Rakib
Back to blog
Laravel
Intermediate

Laravel Queue Workers: Production Setup, Scaling, and Best Practices

Laravel queue workers turn slow background jobs into reliable, asynchronous workflows. This guide covers production-ready configuration, scaling, and observability for real systems.

June 15, 202419 min read

Introduction

Modern web applications cannot afford to make users wait. Sending emails, generating PDFs, processing uploads, syncing third-party APIs, and resizing images are all tasks that should happen in the background. Laravel queue workers are the engine that makes this possible. They consume jobs from a queue connection, execute the associated logic, and report success or failure without blocking an HTTP request.

In development, running php artisan queue:work in a terminal is enough. In production, that approach breaks quickly. Workers crash, memory leaks accumulate, jobs get stuck, and sudden traffic spikes overwhelm a single process. This article explains how to run Laravel queue workers in production with Redis, Supervisor, retries, timeouts, monitoring, and scaling strategies that actually hold up under load.

By the end, you will understand the architecture, be able to deploy a resilient worker setup, and avoid the most common mistakes that cause silent job loss.

Table of Contents

Core Concepts

Laravel queues are built around a few simple but powerful concepts. Understanding them prevents confusion later.

Jobs

A job is a class that implements the ShouldQueue interface or extends Illuminate\Queue\Jobs\Job through the dispatch helper. Each job represents a unit of work. Laravel serializes the job and its data into the queue backend.

Connections and Queues

A connection defines the driver (database, Redis, Amazon SQS, Beanstalkd). A queue is a named channel within that connection. You can route jobs to different queues such as emails, exports, or webhooks.

Workers

A worker is a long-running PHP process started by php artisan queue:work. It listens to one or more queues and executes jobs as they arrive.

Failed Jobs

When a job exceeds its retry limit, Laravel inserts a record into the failed_jobs table so you can inspect and retry it manually.

Supervisor

Supervisor is a process control system for Linux. It restarts crashed workers automatically, manages logs, and keeps your worker pool alive.

Architecture Overview

A typical production Laravel queue architecture looks like this:

  • Web application dispatches jobs during HTTP requests.
  • Jobs are pushed to a Redis queue.
  • Multiple Laravel queue workers pull jobs from Redis.
  • Supervisor monitors each worker process and restarts it if it exits.
  • Failed jobs are stored in the database.
  • Optional: Laravel Horizon provides a dashboard and auto-scaling configuration for Redis queues.

Redis is the recommended production backend because it is fast, supports atomic operations, and handles high throughput with low latency. Database queues are fine for low-volume apps but create extra load on your primary database.

Step-by-Step Guide

1. Install and Configure Redis

On Ubuntu, install Redis:

sudo apt updatesudo apt install redis-serverredis-cli ping

Set a strong password in /etc/redis/redis.conf:

requirepass your-strong-redis-password

2. Configure Laravel Queue Connection

In config/queue.php, set the default connection to Redis and define the connection details in .env:

QUEUE_CONNECTION=redisREDIS_HOST=127.0.0.1REDIS_PASSWORD=your-strong-redis-passwordREDIS_PORT=6379

3. Create the Failed Jobs Table

php artisan queue:failed-tablephp artisan migrate

4. Install Supervisor

sudo apt install supervisor

5. Create a Supervisor Config

Create /etc/supervisor/conf.d/laravel-worker.conf:

[program:laravel-worker]process_name=%(program_name)s_%(process_num)02dcommand=php /var/www/rakibahsan/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600autostart=trueautorestart=truestopasgroup=truekillasgroup=trueuser=www-datanumprocs=4redirect_stderr=truestdout_logfile=/var/www/rakibahsan/storage/logs/worker.logstopwaitsecs=3600

6. Start Supervisor

sudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl start laravel-worker:*

Real-World Examples

Consider an e-commerce platform. When a user places an order, the controller dispatches three jobs:

  • SendOrderConfirmationEmail
  • GenerateInvoicePdf
  • SyncOrderToCrm

Each job runs on a separate queue so a slow CRM sync never delays the confirmation email. The email queue has more workers, while the CRM queue has fewer because the external API is rate-limited.

Another example is a SaaS export feature. A user requests a CSV of 100,000 rows. The HTTP request dispatches GenerateExportJob. The worker streams the data to a file in object storage and emails a download link when finished. The user gets an immediate response instead of a 30-second timeout.

Production Code Examples

Job Class with Retries and Timeout

<?phpnamespace App\Jobs;use Illuminate\Bus\Queueable;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Foundation\Bus\Dispatchable;use Illuminate\Queue\InteractsWithQueue;use Illuminate\Queue\SerializesModels;use App\Mail\OrderConfirmation;use Illuminate\Support\Facades\Mail;class SendOrderConfirmationEmail implements ShouldQueue{    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;    public int $tries = 3;    public int $timeout = 120;    public string $queue = 'emails';    public function __construct(        public int $orderId,        public string $email    ) {}    public function handle(): void    {        Mail::to($this->email)->send(new OrderConfirmation($this->orderId));    }    public function failed(\Throwable $exception): void    {        logger()->error('Order email failed', [            'order_id' => $this->orderId,            'error' => $exception->getMessage(),        ]);    }}

Dispatching with Delay

SendOrderConfirmationEmail::dispatch($order->id, $user->email)    ->onQueue('emails')    ->delay(now()->addSeconds(10));

Supervisor-Managed Worker Command

php artisan queue:work redis --queue=emails,exports,webhooks --sleep=3 --tries=3 --max-time=3600 --memory=512

Retrying Failed Jobs

php artisan queue:retry allphp artisan queue:forget 42php artisan queue:flush

Comparison Table

Choosing the right queue backend affects reliability and cost. Here is a practical comparison.

BackendThroughputSetup ComplexityBest For
DatabaseLowVery LowSmall apps, low job volume
RedisHighMediumProduction, high throughput
Amazon SQSVery HighMediumServerless, AWS-native systems
BeanstalkdMediumLowSimple dedicated queue servers

Best Practices

  • Always use Supervisor or a container orchestrator to keep workers alive.
  • Set --max-time to rotate workers and avoid memory leaks.
  • Separate queues by priority and job type.
  • Use --memory limit to prevent runaway processes.
  • Log failed jobs and alert on spikes.
  • Make jobs idempotent so retries are safe.
  • Use Horizon if you already use Redis and want metrics.
  • Version job classes carefully to avoid serialization errors.

Common Mistakes

  • Running queue:listen in production instead of queue:work.
  • Forgetting to run migrations for the failed jobs table.
  • Not setting a Redis password, exposing the queue to the internet.
  • Using sleep(10) inside jobs instead of queue delays.
  • Storing large objects in job constructors instead of IDs.
  • Ignoring memory growth until the worker is OOM-killed.
  • Putting CPU-heavy jobs on the same queue as user-facing emails.

Performance Tips

  • Increase numprocs based on CPU cores and job type.
  • Use Redis pipelining where possible for batch operations.
  • Keep job payloads small; pass IDs, not full models.
  • Use --timeout slightly higher than job $timeout.
  • Preload config and routes with php artisan config:cache.
  • Offload file processing to object storage and temporary disks.

Security Considerations

  • Redis must require a password and bind to localhost or a private network.
  • Jobs should not contain raw secrets; load them from environment at runtime.
  • Validate all job input to prevent injection via deserialized data.
  • Restrict Supervisor HTTP server or keep it disabled.
  • Use least-privilege database users for worker connections.
  • Audit failed jobs because they may expose stack traces.

Deployment Notes

During deployment, restart workers so they load new code. With Supervisor:

sudo supervisorctl restart laravel-worker:*

If using Zero Downtime Deployment, deploy to a new release directory and point Supervisor to the new path, then restart. For Docker, run workers as a separate service with a restart policy of always.

services:  worker:    build: .    command: php artisan queue:work redis --tries=3 --max-time=3600    restart: always    depends_on:      - redis

Debugging Tips

  • Check Supervisor status: sudo supervisorctl status.
  • Inspect worker logs in storage/logs/worker.log.
  • Use php artisan queue:failed to list failures.
  • Run a single job locally with php artisan queue:work --once.
  • Monitor Redis memory with redis-cli info memory.
  • Enable Laravel Telescope or Horizon for job visibility.

FAQ

What is the difference between queue:work and queue:listen?

queue:work boots the framework once and keeps running, making it ideal for production. queue:listen restarts the framework for every job, which is slower and only useful in local development.

How many workers should I run?

Start with one worker per CPU core and adjust based on job type. I/O-heavy jobs can use more; CPU-heavy jobs should match core count.

Why are my jobs stuck in pending?

Usually the worker is not running, Redis is unreachable, or the queue name does not match. Check Supervisor status and Redis connectivity.

How do I prevent duplicate jobs?

Use unique job keys with ->unique() or check existing unprocessed jobs before dispatch. Make handlers idempotent.

Should I use Horizon or Supervisor?

Use Supervisor for simple control. Use Horizon when you want a dashboard, auto-scaling, and metrics on Redis queues.

What happens if a worker dies mid-job?

Redis retains the job. With proper retry configuration, the job is reattempted by another worker after the timeout.

Can I schedule queues with cron?

No. Workers run continuously. Use Laravel Task Scheduling to dispatch jobs at intervals, not to run the worker itself.

How do I handle jobs that always fail?

Inspect the failed_jobs table, fix the root cause, then use queue:retry. Add monitoring to catch repeated failures early.

Is Redis required for Laravel queues?

No. Database, SQS, Beanstalkd, and sync are supported. Redis is recommended for production due to speed and reliability.

Conclusion

Laravel queue workers are essential for building responsive, scalable applications. In production, never rely on a manual terminal command. Use Redis as your backend, Supervisor to keep processes alive, separate queues for different workloads, and strict retry and timeout policies. Monitor failed jobs and restart workers on deploy. With this setup, your background jobs will be reliable, observable, and ready for real traffic.

If you want a complete, copy-paste Supervisor and Redis configuration for your next Laravel project, subscribe to the blog or check the related articles on Dockerizing Laravel and Redis cache configuration.