Laravel Redis Queues: Scalable Background Job Processing

Introduction

Modern web applications are expected to respond in milliseconds while simultaneously handling resource intensive operations such as sending transactional emails, generating PDF reports, processing uploaded images, synchronizing data with third party platforms, and running machine learning inference. Performing these tasks synchronously inside an HTTP request is a recipe for timeouts, poor user experience, and unreliable systems. The Laravel ecosystem solves this problem elegantly through its queue system, and when paired with Redis as the backing store, it becomes a high throughput, low latency background job engine trusted by startups and enterprise teams alike.

In this comprehensive guide we will explore Laravel Redis queues from first principles to production grade implementations. You will learn the core concepts that differentiate a job from a command, understand the architecture that connects your application to Redis and worker processes, follow a step by step setup that you can replicate in your own project, examine real world use cases, study production ready code examples, compare Redis against alternative queue drivers, and absorb best practices that prevent the most common failures. Whether you are building your first Laravel application or scaling an existing platform to millions of users, the patterns described here will help you design resilient asynchronous workflows.

We assume you have a working Laravel installation, basic familiarity with PHP, and access to a Redis server. If you are new to Redis, think of it as an in memory data structure store that can act as a cache, message broker, and database. Its blazing fast operations make it an ideal transport for queue jobs where latency and throughput matter. By the end of this article you will be able to configure Laravel to dispatch jobs to Redis, run supervised worker processes, monitor failures, and deploy the entire stack with confidence.

Table of Contents

• Introduction
• Core Concepts
• Architecture Overview
• Step-by-Step Guide
• Real-World Examples
• Production Code Examples
• Comparison Table
• Best Practices
• Common Mistakes
• Performance Tips
• Security Considerations
• Deployment Notes
• Debugging Tips
• FAQ
• Conclusion

Core Concepts

Before writing code, it is essential to understand the vocabulary of the Laravel queue system. A job is a class that encapsulates a unit of work. It implements the ShouldQueue interface or is dispatched via the dispatch helper to indicate it should be pushed to a queue rather than executed immediately. A queue is a named pipeline (for example default, emails, reports) that holds jobs until a worker picks them up. A worker is a long running Artisan command (queue:work or queue:listen) that connects to the queue driver, pops jobs, and executes them.

Redis acts as the queue driver. When Laravel dispatches a job, it serializes the job object into a JSON or PHP serialized string and pushes it onto a Redis list or sorted set depending on the configuration. Redis provides atomic operations such as RPUSH and BLPOP which guarantee that even with many workers, each job is delivered to exactly one consumer. This is the foundation of reliable distributed processing.

Laravel also introduces the concept of delayed and scheduled jobs. A delayed job is placed in a Redis sorted set with a score equal to the timestamp when it should become available. The worker periodically checks this set and moves due jobs into the active list. Retries are handled by recording the number of attempts and, on exception, re-enqueuing the job with a backoff delay. Failed jobs are stored in a database table (failed_jobs) so you can inspect and retry them later.

Another important concept is queue priority. Redis driver supports multiple queues per worker. You can start a worker with --queue=high,default so that the high queue is always processed before the default queue. This enables you to prioritize user facing notifications over bulk exports. Finally, Horizon is a first party Laravel package that provides a beautiful dashboard and configuration layer for Redis queues, allowing you to define worker pools, auto scale, and monitor throughput in real time.

Architecture Overview

A typical Laravel Redis queue architecture consists of four logical components: the application server, the Redis server, the worker processes, and the persistence layer for failed jobs. The application server runs your Laravel web nodes that handle HTTP requests. When a controller needs to send a welcome email, it dispatches a SendWelcomeEmail job. Laravel serializes the job and issues a Redis command to push it onto the queues:default list.

The Redis server is a standalone or clustered instance accessible over the network. It holds the jobs in memory, providing sub millisecond push and pop operations. Because Redis is single threaded for command execution, it can handle tens of thousands of operations per second, easily saturating your worker capacity before becoming a bottleneck. For high availability you can deploy Redis with replication and Sentinel or use a managed Redis service.

Worker processes run on one or more machines. Each worker is a PHP process spawned by Supervisor or Kubernetes that executes php artisan queue:work redis. The worker maintains a persistent connection to Redis, blocks on the pop operation, and once a job arrives, deserializes it, resolves its dependencies via the Laravel container, and invokes the handle method. After success, the job is removed from the queue. If it fails, the worker increments the attempt count and either retries or records the failure.

The persistence layer is usually your primary MySQL or PostgreSQL database where Laravel stores the failed_jobs table and migration state. This separation ensures that even if Redis restarts and loses its in memory data (without persistence enabled), you still have a record of jobs that could not be processed. In production, enable Redis AOF or RDB persistence to minimize job loss. The flow is described below:

• Web request -> dispatch job -> Redis list
• Worker -> BLPOP from Redis -> execute -> success/fail
• Fail -> failed_jobs table in database

This decoupled design allows your web tier to scale independently of your worker tier, a core principle of modern software architecture.

Step-by-Step Guide

Let us walk through a complete setup of Laravel Redis queues on a fresh project. The steps assume Laravel 10 or later, PHP 8.1+, and a Redis server reachable at 127.0.0.1:6379.

1. Install Laravel and Redis PHP extension

Ensure the redis PHP extension or the predis/predis composer package is available. Laravel 10 uses the phpredis extension by default. Install it via PECL or your system package manager. Then create a project:

composer create-project laravel/laravel redis-queue-democd redis-queue-demo

2. Configure Redis connection

Open .env and set the queue connection and Redis credentials:

REDIS_HOST=127.0.0.1REDIS_PASSWORD=nullREDIS_PORT=6379QUEUE_CONNECTION=redis

The config/queue.php file already contains a redis entry pointing to the default Redis connection. You can define multiple queue names under config/queue.php if needed.

3. Create a job class

Generate a job that simulates sending a notification:

php artisan make:job SendWelcomeNotification

Then edit app/Jobs/SendWelcomeNotification.php as shown later in the production code section.

4. Dispatch the job

In a controller or route, dispatch the job:

use App\Jobs\SendWelcomeNotification;use Illuminate\Support\Facades\Route;Route::post('/register', function (Request $request) {    // create user logic...    SendWelcomeNotification::dispatch($user)->onQueue('emails');    return response()->json(['message' => 'User registered']);});

5. Start a worker

Run the worker locally to process jobs:

php artisan queue:work redis --queue=emails

You should see the worker start, connect to Redis, and process any dispatched jobs. For production, you will use Supervisor to keep this process alive, which we cover in deployment notes.

6. Test the flow

Use a tool like curl or Postman to hit the registration endpoint, then observe the worker output. The job should execute within milliseconds and log confirmation. If it fails, check the failed_jobs table after running the migration php artisan queue:failed-table and php artisan migrate.

Real-World Examples

Laravel Redis queues shine in many production scenarios. Below are four common use cases drawn from real systems.

1. Transactional Email Sending: Instead of calling an SMTP or HTTP email API during a request, dispatch a SendEmailJob. This keeps page load under 100ms even when the email provider is slow.

2. Image and Video Processing: User uploads a profile picture. The web node stores the raw file and dispatches a ProcessImageJob that generates thumbnails, applies watermarks, and uploads to S3. The user sees instant confirmation while processing happens asynchronously.

3. Report Generation: A dashboard requests a monthly PDF sales report. The controller dispatches GenerateReportJob that queries the database, renders a view, and stores the PDF in object storage. When done, it notifies the user via a second queued notification.

4. Third-Party Sync and Webhooks: When a payment succeeds in Stripe, your webhook endpoint dispatches a SyncInvoiceJob. If the external API is rate limited, the job retries with exponential backoff, ensuring eventual consistency without blocking the webhook response.

These examples share a pattern: the web request does the minimal work to capture intent, and the heavy lifting is delegated to workers. This leads to systems that degrade gracefully under load.

Production Code Examples

The following snippets illustrate a robust implementation suitable for production. First, the job class with proper serialization, retry, and timeout settings.

<?phpnamespace App\Jobs;use App\Models\User;use Illuminate\Bus\Queueable;use Illuminate\Contracts\Queue\ShouldQueue;use Illuminate\Foundation\Bus\Dispatchable;use Illuminate\Queue\InteractsWithQueue;use Illuminate\Queue\SerializesModels;use Illuminate\Support\Facades\Mail;use App\Mail\WelcomeMail;class SendWelcomeNotification implements ShouldQueue{    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;    public int $tries = 3;    public int $maxExceptions = 2;    public int $timeout = 60;    public array $backoff = [5, 15, 30];    public function __construct(public User $user)    {    }    public function handle(): void    {        Mail::to($this->user->email)->send(new WelcomeMail($this->user));    }    public function failed(\Throwable $exception): void    {        logger()->error('Welcome email failed for user '.$this->user->id, [            'error' => $exception->getMessage(),        ]);    }}

Notice the use of public User $user with the SerializesModels trait. Laravel will store only the model's identifier in Redis and re-fetch the model when the job is handled, reducing payload size and avoiding stale state.

Next, a controller dispatching multiple prioritized jobs:

public function store(Request $request){    $user = User::create($request->validated());    SendWelcomeNotification::dispatch($user)->onQueue('high');    GenerateProfileRecommendations::dispatch($user)->onQueue('low');    return redirect()->route('home');}

To run workers with prioritized queues, use:

php artisan queue:work redis --queue=high,low --tries=3 --timeout=60

If you adopt Laravel Horizon, the configuration lives in config/horizon.php:

'environments' => [    'production' => [        'supervisor-1' => [            'connection' => 'redis',            'queue' => ['high', 'low'],            'maxProcesses' => 10,            'balance' => 'auto',        ],    ],],

Finally, a Supervisor configuration snippet to keep workers alive:

[program:laravel-worker]process_name=%(program_name)s_%(process_num)02dcommand=php /var/www/html/artisan queue:work redis --queue=high,low --sleep=3 --tries=3autostart=trueautorestart=trueuser=www-datanumprocs=4redirect_stderr=truestdout_logfile=/var/log/laravel-worker.log

Comparison Table

Choosing the right queue driver depends on your infrastructure and scale. The table below compares the most common Laravel queue backends.

Redis offers the best balance of speed, operational simplicity, and feature richness when you already use it for caching. Its sorted set support enables precise delayed job scheduling without polling overhead.

Best Practices

• Make jobs idempotent: Workers can process a job more than once due to retries. Ensure your handle method can run multiple times without side effects, using database unique constraints or locks.
• Keep payloads small: Pass model IDs instead of entire objects. Use the SerializesModels trait to automatically reference models.
• Set explicit timeouts: A runaway job should not block a worker forever. Use $timeout and --timeout flag.
• Use dedicated queues: Separate fast user actions from slow batch jobs to avoid head-of-line blocking.
• Monitor with Horizon: Horizon provides real-time metrics, job timelines, and failure alerts.
• Handle failures gracefully: Implement the failed method to alert your team and store context.
• Batch related jobs: Use Bus::batch for operations that should succeed or fail together, such as importing a CSV with multiple rows.
• Version your jobs: When changing job structure, maintain backward compatibility or write a migration to clear old jobs.

Common Mistakes

• Dispatching non-serializable data: Passing a resource like a file handle or a Doctrine connection will fail. Only pass primitives or eloquent models.
• Running queue:listen in production: listen restarts the framework on every job, wasting resources. Use queue:work with Supervisor.
• Not setting retry backoff: Immediate retries can hammer a failing external API. Define $backoff array.
• Ignoring memory leaks: Long running workers accumulate memory. Use --max-time or --max-jobs to recycle processes.
• Blocking Redis with large payloads: Storing multi-megabyte JSON in Redis increases latency for all queues. Offload large data to object storage.
• Forgetting failed_jobs migration: Without the table, failed jobs vanish. Always run php artisan queue:failed-table and migrate.

Performance Tips

• Run multiple worker processes across cores; PHP is single threaded per process, so concurrency requires many processes.
• Use Horizon's balance setting to dynamically allocate workers to busy queues.
• Tune Redis maxmemory-policy to avoid eviction of jobs under memory pressure.
• Disable blocking pop if you have many empty queues; use sleep interval to reduce CPU.
• Batch database writes inside jobs using transactions or bulk inserts.
• Compress job payloads if they contain repetitive text by using a custom queue connector or compressing before dispatch.
• Monitor job throughput with Redis SLOWLOG to detect expensive commands.

Security Considerations

• Protect Redis: Bind Redis to private network, require a password, and disable dangerous commands like FLUSHALL in production.
• Isolate worker network: Workers need Redis and database access but should not be publicly reachable.
• Sanitize job data: Jobs often carry user input. Validate before dispatch and escape when using in notifications.
• Avoid secret leakage: Never store plain text passwords or API keys in job properties. Fetch from environment at handle time.
• Use least privilege DB users: Worker database connections should have only necessary table permissions.
• Audit failed jobs: Failed job payloads may contain sensitive data; restrict access to the failed_jobs table.

Deployment Notes

When deploying to a Linux server, install Supervisor and create the configuration shown earlier. For Docker, create a service that runs the same Artisan command:

services:  app:    build: .    command: php-fpm  worker:    build: .    command: php artisan queue:work redis --queue=high,low    depends_on:      - redis  redis:    image: redis:7-alpine

On Kubernetes, deploy workers as a Deployment with a replica count matching your concurrency needs, and use liveness probes that check the worker log or a heartbeat. Always set QUEUE_CONNECTION=redis in the environment, and consider using a dedicated Redis instance for queues separate from cache to avoid cache eviction affecting jobs.

Enable Redis persistence (appendonly yes) to survive restarts. For zero data loss, combine Redis with the database failed jobs table and a dead-letter queue pattern: after max retries, move job to a dead queue for manual inspection.

Debugging Tips

• Use php artisan queue:failed to list failed jobs and queue:retry to replay them.
• Run a single job synchronously with --once and --verbose to see stack traces.
• Monitor Redis directly: redis-cli monitor shows push/pop commands in real time (use only in staging).
• Check Supervisor logs at /var/log/laravel-worker.log for PHP fatal errors.
• Add contextual logging inside handle using a correlation ID passed via job property.
• If jobs stay queued, verify worker is connected to same Redis database index as dispatcher.
• Use Horizon's "recent jobs" tab to see execution time and memory usage.

FAQ

What is the difference between queue:work and queue:listen?

queue:work boots the Laravel framework once and processes many jobs in a persistent process, making it efficient for production. queue:listen restarts the framework for every job, which is useful during development but consumes more CPU and memory. Always prefer queue:work in production with a process manager like Supervisor.

Can I use Redis clustering with Laravel queues?

Yes, Laravel supports Redis clustering, but be aware that some queue operations rely on blocking pop commands that may not be supported across cluster slots. For most setups, a single Redis instance or a master-replica with Sentinel is simpler and sufficient. If you need horizontal scale, consider Horizon with multiple Redis instances and separate queue names.

How do I prevent duplicate job execution?

Design jobs to be idempotent. Use database unique constraints, cache locks (Cache::lock), or Laravel's WithoutOverlapping middleware to ensure only one instance of a job runs for a given key. Also, set appropriate $tries and $backoff to avoid rapid repeats.

Why are my jobs delayed even when Redis is fast?

Common causes include insufficient worker processes, long job execution time, or a misconfigured sleep interval. If you dispatch with delay or later, that is expected. Also check if the worker is processing a lower priority queue while high priority jobs wait. Use --queue=high,default ordering.

How do I handle jobs that exceed max retries?

After exhausting retries, Laravel inserts the job into the failed_jobs table. You can inspect the exception, fix the underlying issue, and run php artisan queue:retry with the job ID. For automated recovery, implement a scheduled command that scans failed jobs and retries based on rules.

Is it safe to pass Eloquent models to jobs?

Yes, when using the SerializesModels trait, Laravel stores only the model's primary key and re-queries the database when the job is handled. This keeps the Redis payload tiny and ensures you work with fresh data. However, if the model is deleted before the job runs, you must handle a ModelNotFoundException.

How can I monitor queue health in production?

Use Laravel Horizon for a rich dashboard showing throughput, wait time, and failures. For external monitoring, export Redis metrics to Prometheus, set alerts on queue length, and use logging aggregation (e.g., Laravel Logs to Papertrail or ELK) to track job errors. A simple cron can also ping a health endpoint that reports pending job counts.

Can I run queue workers on serverless platforms?

Traditional long-running workers do not fit AWS Lambda's execution model, but you can use Laravel Vapor or a container-based Fargate task that runs queue:work inside a container. Alternatively, use SQS as the driver and process with Lambda consumers. Redis based workers require a persistent runtime, so choose EC2, DigitalOcean Droplets, or Kubernetes for stateful processing.

Conclusion

Laravel Redis queues provide a battle tested path to building responsive, scalable applications. By offloading heavy work to background workers, you protect user experience and unlock architectural patterns such as event driven microservices, batch processing, and resilient third party integrations. Start by configuring Redis as your queue driver, write small focused job classes, supervise your workers, and monitor with Horizon.

If you found this guide useful, explore our related articles on Laravel Horizon and Redis performance tuning linked above. Implement queues in your next feature and watch your request times drop while reliability climbs. Have a specific use case? Dispatch your first job today and join the thousands of developers shipping faster with Laravel and Redis.