Mastering Laravel Queues: A Complete Guide to Background Job Processing
Laravel's queue system is one of its most powerful features, allowing developers to defer time-consuming tasks and improve application performance. Whether you're sending emails, processing images, or generating reports, queues can help you build more responsive applications.
What Are Laravel Queues?
Laravel queues provide a unified API for various queue backends, allowing you to push time-consuming tasks to the background. This means your web requests can respond quickly while heavy lifting happens behind the scenes.
Why Use Queues?
1. Improved User Experience
Instead of making users wait for slow operations, you can queue them and provide immediate feedback.
2. Better Performance
Offload CPU-intensive tasks to background workers, keeping your web interface responsive.
3. Fault Tolerance
Failed jobs can be retried automatically, ensuring reliability.
4. Scalability
Run multiple queue workers across different servers to handle increased load.
Setting Up Laravel Queues
Configuration
First, configure your queue driver in .env:
QUEUE_CONNECTION=database
# or redis, sqs, etc.
For database queues, run the migration:
php artisan queue:table
php artisan migrate
Creating Jobs
Generate a new job class:
php artisan make:job ProcessPayment
This creates a job class in app/Jobs:
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
class ProcessPayment implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $payment;
public function __construct($payment)
{
$this->payment = $payment;
}
public function handle()
{
// Process the payment
$paymentGateway = new PaymentGateway();
$paymentGateway->charge($this->payment);
// Send confirmation email
Mail::to($this->payment->user)->send(new PaymentConfirmation($this->payment));
}
}
Dispatching Jobs
Basic Dispatching
// Dispatch immediately
ProcessPayment::dispatch($payment);
// Dispatch with delay
ProcessPayment::dispatch($payment)->delay(now()->addMinutes(5));
// Dispatch to specific queue
ProcessPayment::dispatch($payment)->onQueue('payments');
Conditional Dispatching
// Only dispatch if condition is met
ProcessPayment::dispatchIf($payment->requires_processing, $payment);
// Dispatch unless condition is met
ProcessPayment::dispatchUnless($payment->is_processed, $payment);
Queue Workers and Management
Running Queue Workers
Start processing jobs:
# Process jobs from default queue
php artisan queue:work
# Process specific queue
php artisan queue:work --queue=payments,emails
# Process jobs with timeout
php artisan queue:work --timeout=60
Supervisor Configuration
For production, use Supervisor to manage queue workers:
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/path/to/your/app/storage/logs/worker.log
Advanced Queue Features
Job Batching
Process related jobs together:
use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;
$batch = Bus::batch([
new ProcessPayment($payment1),
new ProcessPayment($payment2),
new ProcessPayment($payment3),
])->then(function (Batch $batch) {
// All jobs completed successfully
})->catch(function (Batch $batch, Throwable $e) {
// First batch job failure detected
})->finally(function (Batch $batch) {
// The batch has finished executing
})->dispatch();
Job Middleware
Create reusable job logic:
<?php
namespace App\Jobs\Middleware;
class RateLimited
{
public function handle($job, $next)
{
Redis::throttle('key')
->block(0)->allow(1)->every(5)
->then(function () use ($job, $next) {
$next($job);
}, function () use ($job) {
$job->release(5);
});
}
}
Apply middleware to jobs:
public function middleware()
{
return [new RateLimited];
}
Failed Jobs
Handle job failures gracefully:
public function failed(Exception $exception)
{
// Send user notification of failure, etc..
Log::error('Payment processing failed', [
'payment_id' => $this->payment->id,
'error' => $exception->getMessage()
]);
}
Retry failed jobs:
# Retry all failed jobs
php artisan queue:retry all
# Retry specific job
php artisan queue:retry 5
Monitoring and Optimization
Queue Monitoring
Use Laravel Horizon for Redis queues:
composer require laravel/horizon
php artisan horizon:install
php artisan horizon
Performance Tips
- Use appropriate queue drivers - Redis for high-throughput, database for simplicity
- Set proper timeouts - Prevent jobs from running indefinitely
- Monitor memory usage - Restart workers periodically
- Use job batching - Process related jobs efficiently
- Implement proper error handling - Log failures and notify administrators
Queue Configuration Best Practices
// config/queue.php
'connections' => [
'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => env('REDIS_QUEUE', 'default'),
'retry_after' => 90,
'block_for' => null,
],
],
Real-World Examples
Email Notifications
class SendWelcomeEmail implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $user;
public function __construct(User $user)
{
$this->user = $user;
}
public function handle()
{
Mail::to($this->user)->send(new WelcomeEmail($this->user));
}
}
// Usage
SendWelcomeEmail::dispatch($user);
Image Processing
class ProcessUploadedImage implements ShouldQueue
{
public $image;
public function handle()
{
$image = Image::make($this->image->path);
// Create thumbnails
$image->resize(300, 200)->save(storage_path('thumbnails/' . $this->image->name));
// Optimize original
$image->encode('jpg', 85)->save();
}
}
Conclusion
Laravel queues are essential for building scalable, responsive applications. By moving time-consuming tasks to the background, you can provide better user experiences while maintaining system performance.
Start with simple database queues for development, then move to Redis or SQS for production. Remember to monitor your queues, handle failures gracefully, and scale your workers based on demand.
The investment in setting up a proper queue system will pay dividends as your application grows and handles more complex background tasks.
What's your experience with Laravel queues? Share your tips and challenges in the comments below!
Top comments (0)