Background workers
Process queues and long-running asynchronous work outside your HTTP server.
A background worker runs a continuous process without a public URL. It can consume queued emails, generate reports, process media, or perform other work that should not keep a user's HTTP request open.
Create a worker
- Create a Background Worker in your project.
- Select its source. A web service and worker can use the same repository with different start commands.
- Configure the build, then set the command that runs the worker in the foreground.
- Add database, queue, and application variables. Use private connections for Openstead databases and Key Value.
- Select a paid plan and deploy.
Example start commands:
| Application | Example |
|---|---|
| Celery | celery -A config worker --loglevel=info |
| Laravel | php artisan queue:work --sleep=3 --tries=3 --timeout=90 |
| Node.js | node dist/worker.js |
| Sidekiq | bundle exec sidekiq |
Replace module names and paths with the ones in your application. A worker should keep running; do not daemonize it or send the main process into the background.
Make jobs safe to repeat
A process can stop after it has changed data but before it acknowledges a queue message. Make each job idempotent: store a job identifier, check whether the work has already completed, and use database transactions where appropriate.
Configure retries, retry delays, time limits, and dead-letter handling in your queue library. Openstead running the worker does not configure those application-level policies for you.
Connect a queue
For a Redis-compatible queue, create Key Value and supply its private connection URL to both the producer and worker. Select an eviction policy appropriate for the queue; evicting an unprocessed queue entry can lose work.
For Laravel's database queue, the worker and web service need access to the same intended database. Run the queue schema migrations before accepting jobs.
Deploy without losing track of work
Handle termination gracefully and give unfinished messages back to the queue. Release long-running database connections when exiting. Inspect Logs after a deployment to confirm the worker actually connects and consumes work; a running process alone does not prove that jobs are being completed.
Stateless workers can use scaling. Confirm that adding consumers is safe for the queue and database connection pool. Use a cron job for a command that should start on a schedule, complete, and exit.