Case · Go backend
Conveyor: job queue
A background job queue: priorities, delayed and cron jobs, retries with backoff and jitter, dead-letter, idempotency keys, leases with fencing, Prometheus metrics.
- Go
- SQLite
- REST API
- SSE
- Prometheus
- Docker
01 · How it looks
Screenshots
02 · About
What it is and why
Conveyor is a background job queue: persistent in SQLite, with a worker pool, priorities, delayed and cron jobs, retries with exponential backoff and jitter, and a dead-letter queue with manual retry.
It guarantees at-least-once delivery: a job is held under a lease with a heartbeat, and the lease token (fencing) stops a zombie worker from corrupting the result. It also has Prometheus metrics and a live SSE dashboard.
Limitations
The demo handlers (email.send, image.resize, webhook.call) only simulate work: nothing is sent.
A single SQLite file, so it scales only within one machine. At-least-once only, so handlers must be idempotent; there are no job chains and no per-queue rate limits. Metric counters reset on restart.
What is inside
- Priorities 0-9, delayed start, and a custom cron expression parser (lists, ranges, steps, @daily and so on)
- Idempotency keys: enqueueing again returns the existing job, even when two requests race
- Atomic job claiming with a single UPDATE ... RETURNING, so no double execution even from two processes
- Lease and reaper for crashed workers; graceful shutdown returns unfinished jobs to the queue
- REST API, /metrics in Prometheus format, a dashboard
- Tests with -race for concurrent claiming, lease, backoff, cron and shutdown; Docker
03 · How to run
Run it locally
git clone <repository on request> job-queue
cd job-queue
API_TOKEN=demo SEED_DEMO=1 go run ./cmd/server # http://localhost:808504 · Code
The repository is private
The source lives in a private repository. I will grant access on request: write to me and I will send an invitation.
Request the code