Batch, Bulk API
The Batch Bulk API (more commonly called the Bulk API, with Bulk API 2.0 as the modern version) is the Salesforce REST-based interface designed for high-volume asynchronous data operations: insert,…
Definition
The Batch Bulk API (more commonly called the Bulk API, with Bulk API 2.0 as the modern version) is the Salesforce REST-based interface designed for high-volume asynchronous data operations: insert, update, upsert, delete, hard delete, and query against tens of thousands to hundreds of millions of records. Where the regular REST API handles one or a few records per call, Bulk API accepts a CSV (or JSON) payload of many records at once, processes them in chunks on Salesforce's async layer, and returns success/failure results that the client polls for. Bulk API 2.0 simplifies job lifecycle compared to 1.0 and is the recommended version for new integrations.
Bulk API exists because the regular REST API governor limits make large data loads impractical. Loading a million Account records through standard REST would burn the daily API quota and take hours. Bulk API processes the same job in minutes by running batches against larger limits on the async layer. Data Loader, MuleSoft, Informatica, and most ETL tools use Bulk API under the hood for any operation above a few thousand records. Reporting tools that need bulk export use Bulk API Query for the same reason. The trade-off is async semantics: clients submit a job and poll for completion rather than receiving immediate responses.
How the Bulk API moves data in and out of Salesforce
Bulk API 1.0 versus 2.0
Bulk API 1.0 (introduced around 2010) requires the client to chunk the payload into batches of 10,000 records or 10 MB. Bulk API 2.0 (2018) handles chunking internally: the client uploads one payload of any size and Salesforce splits it. 2.0 is the recommended version for new development; 1.0 remains supported for legacy integrations but is no longer the default for new Data Loader-style clients.
The job lifecycle
Bulk API 2.0 jobs progress through Open, UploadComplete, InProgress, JobComplete, Failed, Aborted. The client creates a job, uploads the CSV payload, closes the upload, and polls for status. When complete, the client downloads the successful and failed record results. Each phase has its own REST endpoint.
Supported operations
Bulk API supports Insert, Update, Upsert (with an external ID), Delete, Hard Delete (skipping the Recycle Bin), and Query (with optional cursor for sets above the in-memory limit). The Query operation returns a large result set as a CSV. The DML operations accept a CSV and return one row per submitted record with success or error.
Async semantics and polling
Bulk API is asynchronous. The client submits the job and immediately gets a Job ID. The Salesforce platform processes batches on the async layer; the client polls the Job status endpoint until JobComplete or Failed. Most clients poll every 30 to 60 seconds; large jobs can take minutes to hours.
Limits and quotas
Bulk API has its own quota separate from regular REST API limits. Each org gets a daily allowance of Bulk API records (typically 5 million to 150 million depending on edition). Each batch can process up to 10,000 records (1.0) or arbitrary sizes (2.0). Bulk API consumption shows up in the API Usage Last 7 Days dashboard.
PK Chunking for very large queries
Queries against tables with hundreds of millions of records can exceed Bulk API memory. PK Chunking automatically splits the query by primary key range, processing each chunk separately. Enable PK Chunking through an HTTP header on the job; the platform handles the rest.
Error handling and idempotency
Bulk API returns success or error per row. Common errors include validation rule failures, duplicate detection, and missing required fields. Production integrations build retry logic that re-submits only the failed rows, ideally upserting against an external ID to guarantee idempotency.
When to use Bulk versus REST or SOAP
Bulk API is right for any operation above a few thousand records. REST is right for synchronous CRUD on a few records. SOAP API still exists but is legacy. Composite REST handles dozens of records in one transaction. Picking the wrong API for the data volume is the most common integration design mistake.
How to use Bulk API 2.0 for a data load
Bulk API 2.0 requires a client (Data Loader, MuleSoft, custom code, or the Salesforce CLI). The mechanics are straightforward; the discipline is in chunking, idempotency, and monitoring.
- Authenticate against the Salesforce API
Use OAuth (preferably JWT Bearer flow) to obtain an access token. The token authenticates every Bulk API call.
- Create a job
POST to /services/data/vNN.0/jobs/ingest with the operation, object, and content type. Salesforce returns a Job ID.
- Upload the CSV payload
PUT the CSV content to the job's upload endpoint. Bulk API 2.0 handles chunking internally; the client uploads as one payload.
- Close the job
PATCH the job state to UploadComplete. Salesforce begins processing the records on the async layer.
- Poll status and download results
Poll the job status endpoint every 30 to 60 seconds. When JobComplete, GET the successful and failed record results as CSV.
- Bulk API operations are async. Synchronous behaviour requires the client to poll until completion.
- Each row can succeed or fail independently. Production integrations must download the failed-row CSV and retry only those.
- Use Upsert with an external ID for idempotency. Insert + Update logic written by hand is error-prone at bulk scale.
- Very large queries (hundreds of millions of records) need PK Chunking enabled via HTTP header. Without it, the query may fail with memory errors.
Trust & references
Cross-checked against the following references.
- Bulk API 2.0 Developer GuideSalesforce Developer Docs
- Bulk API 1.0 Developer GuideSalesforce Developer Docs
Straight from the source - Salesforce's reference material on Batch, Bulk API.
- Data LoaderSalesforce Help
About the Author
Dipojjal Chakrabarti is a B2C Solution Architect with 29 Salesforce certifications and over 13 years in the Salesforce ecosystem. He runs salesforcedictionary.com to help admins, developers, architects, and cert/interview candidates sharpen their fundamentals. More about Dipojjal.
Test your knowledge
Q1. What is a Batch in the context of the Bulk API?
Q2. What is the maximum number of records per Batch in Bulk API 1.0?
Q3. What is the main reason to use the Bulk API instead of REST or SOAP?
Discussion
Loading discussion…