Definition
A Batch in the Bulk API is a set of records sent in a single request within a Bulk API job. Each job can contain multiple batches, and each batch can hold up to 10,000 records (Bulk API 1.0). Batches are processed asynchronously on the server, and their status (Queued, InProgress, Completed, Failed) can be monitored via the API. The Bulk API is designed for loading or deleting large datasets efficiently.
Real-World Example
a Salesforce developer at CodeBridge recently implemented Batch, Bulk API to create a robust integration between Salesforce and an external system. Using Batch, Bulk API, the developer builds an efficient solution that syncs data in near real-time, handles error scenarios gracefully, and includes detailed logging for troubleshooting.
Why Batch, Bulk API Matters
In the Bulk API, a Batch is a set of records submitted together as part of a larger Bulk API Job. A Job is the top-level container for a bulk data operation (insert, update, upsert, delete, or query), and each Job can contain multiple Batches. In Bulk API 1.0, each Batch can hold up to 10,000 records, and each Job can contain up to 10,000 Batches. Batches are processed asynchronously on the server, and their state (Queued, InProgress, Completed, Failed, NotProcessed) can be polled via the API.
The Bulk API is specifically designed for high-volume data operations that would be inefficient or impossible through the REST or SOAP APIs. By breaking a large dataset into batches, the platform can process them in parallel and manage server resources efficiently. Bulk API 2.0 simplifies this further by handling batch management internally, so the caller just submits the data and the API decides how to split it. Both versions are common in data migration, ETL pipelines, and backup/archive workflows.
How Organizations Use Batch, Bulk API
- •CodeBridge — Runs nightly data loads from their data warehouse into Salesforce using Bulk API 1.0. The ETL pipeline splits data into batches of 5,000 records, submits them all as part of a single Job, and monitors the Job until every batch completes before marking the night's load successful.
- •TerraForm Tech — Migrated 20 million records from a legacy system into Salesforce using the Bulk API. Batches of 10,000 records were submitted in parallel, and the full migration took about 6 hours where a REST-based approach would have taken days.
- •Quantum Labs — Uses Bulk API 2.0 for a recurring archive job. They submit all records to be archived as a single CSV and let the API manage batching internally, which is simpler than Bulk API 1.0 but less flexible for monitoring individual batches.