🛠️ Laravel Queue and Job System: From Table Creation to Production Deployment

As developers, background jobs often become our best friend - handling time-consuming tasks like sending emails, processing videos, or pushing notifications while we chill on the main thread. Laravel makes this super easy using its Queue system.

In this post, we’ll take a step-by-step tour of Laravel’s job handling process, from table creation to production setup with Supervisor. Whether you’re debugging failed jobs or scheduling retries with delays, you’ll find everything you need right here.

📦 Step 1: Creating Required Queue Tables in Laravel

When we start working with queues in Laravel, there are three main tables we usually need:

php artisan queue:table           # Main jobs table
php artisan queue:failed-table    # Failed jobs table
php artisan queue:batches-table   # For batch processing
php artisan migrate               # Run migrations

This will create:

• jobs – Stores pending jobs
• failed_jobs – Stores failed jobs
• job_batches – Stores job batches (for grouped jobs)

🔍 Step 2: Monitoring Failed Jobs

Sometimes jobs fail (bad internet, missing config, DB timeout—you name it). Laravel stores these failed jobs in the failed_jobs table.

🔎 See failed jobs:

php artisan queue:failed

You’ll see a list of all the jobs that failed, including their ID, connection, exception, and timestamp.

🔁 Step 3: Retrying Failed Jobs

You have two options:

1️⃣ Retry a single failed job by ID:

php artisan queue:retry 5   # Replace 5 with your job's ID

🔁 Retry all failed jobs:

php artisan queue:retry all

Perfect for quick recovery during testing or live debugging.

🧹 Step 4: Deleting Failed Jobs

Sometimes you just want a clean slate:

php artisan queue:flush

This deletes all failed jobs from the failed_jobs table.

⏰ Step 5: Automatically Prune Old Failed Jobs

You can also clean up failed jobs older than a certain number of hours. For example:

php artisan queue:prune-failed            # Default is 24 hours
php artisan queue:prune --hours=48        # Deletes jobs older than 48 hours

Set this up in your scheduler for automatic cleanup.

🔄 Step 6: Handling Job Retries with Backoff

Laravel allows you to retry failed jobs with specific retry logic.

Try each job 2 times before failing it:

php artisan queue:work --tries=2

On a specific queue:

php artisan queue:work --queue=notification --tries=2

Add delay between retries (e.g., 1200 seconds = 20 minutes):

php artisan queue:work --queue=notification --tries=2 --backoff=1200

Multiple backoff durations:

First try: wait 10s, second try: wait 20s:

php artisan queue:work --queue=notification --tries=2 --backoff=10,20

🧬 Step 7: Why Use Job Batches?

Imagine you’re sending thousands of emails to users. You can group them into a batch, and track their completion status as a group.

Real-life example:
You’re launching a new course and want to notify 10,000 users. With job batching:

• You track how many emails were sent successfully
• Retry the whole batch if needed
• Notify admin when the whole batch finishes

It gives you clean, trackable, retryable job management.

🚀 Step 8: Running Queues in Production Using Supervisor

In production, we can’t run php artisan queue:work in a terminal manually. That’s where Supervisor comes in.

It’s a process manager that will automatically restart your queue workers if they crash.

🧩 Sample Supervisor Config

Here’s a simple Supervisor config to manage your Laravel queue workers:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work --tries=3 --backoff=30 --timeout=120 --max-jobs=1000 --max-time=3600 --sleep=3 --reset=1
autostart=true
autorestart=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/var/log/supervisor/laravel-worker.log

Place this config in: /etc/supervisor/conf.d/laravel-worker.conf
Then run:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-worker:*