How would you build a queue-based worker pattern in Apex?

"Queue-based worker" means: producers add work items to a queue; workers process them asynchronously, in order or with priorities.

Salesforce-native implementation:

Job Queue Object — a custom Job_Queue__c table with fields:

• Type__c — job type (PROCESS_PAYMENT, SYNC_USER, etc.).
• Payload__c — JSON-serialised input data.
• Status__c — Pending / Running / Completed / Failed / Retry.
• Priority__c — 1 (highest) to 10.
• Scheduled_For__c — when to run.
• Retry_Count__c, Max_Retries__c.
• Error__c — last error.

Producer: anywhere in the system creates a Job_Queue__c record.

Worker: a scheduled Queueable that picks up Pending jobs and runs them.

`apex public class JobWorker implements Queueable, Database.AllowsCallouts { public void execute(QueueableContext ctx) { List<Job_Queue__c> pending = [ SELECT Id, Type__c, Payload__c, Retry_Count__c FROM Job_Queue__c WHERE Status__c = 'Pending' AND Scheduled_For__c <= :DateTime.now() ORDER BY Priority__c, CreatedDate LIMIT 10 ];

if (pending.isEmpty()) return; // nothing to do

for (Job_Queue__c job : pending) { job.Status__c = 'Running'; } update pending;

for (Job_Queue__c job : pending) { try { JobDispatcher.dispatch(job.Type__c, job.Payload__c); job.Status__c = 'Completed'; } catch (Exception e) { job.Retry_Count__c++; job.Error__c = e.getMessage(); job.Status__c = job.Retry_Count__c >= job.Max_Retries__c ? 'Failed' : 'Pending'; job.Scheduled_For__c = DateTime.now().addMinutes(2 * job.Retry_Count__c.intValue()); // exponential backoff } } update pending;

// Chain self if more work pending if (![SELECT count() FROM Job_Queue__c WHERE Status__c = 'Pending'] > 0) { return; } if (Limits.getQueueableJobs() < Limits.getLimitQueueableJobs()) { System.enqueueJob(new JobWorker()); } } } `

Schedulable kicks off worker:

apex public class JobScheduler implements Schedulable { public void execute(SchedulableContext ctx) { System.enqueueJob(new JobWorker()); } }

Schedule every minute via System.schedule.

Why this pattern:

• Decoupled producers and consumers — producers don't wait for the work.
• Retries with backoff — transient failures don't lose work.
• Priority — urgent jobs go first.
• Auditability — every job has a record showing inputs, status, errors.
• Visibility — admins can monitor the queue, manually retry, or cancel.
• Throttling — LIMIT 10 per worker run controls per-job limits.

Advanced features:

• Distributed workers — multiple Schedulables / Queueables; coordinate via record locking (FOR UPDATE).
• Dead-letter queue — failed jobs go to a separate object for manual review.
• Pub/Sub variant — replace polling with Platform Events.

When NOT to do this:

• For simple one-off async, just use Queueable directly.
• For high-volume needs (millions/day), external systems (AWS SQS, Kafka) likely beat Salesforce-native.

This pattern is foundational for any non-trivial async architecture in Salesforce.