ChangeDataCapture: missing ChangeEventHeader / cannot replay events

Your downstream warehouse subscribes to Change Data Capture events from Salesforce to keep an Account replica in sync. A Kafka consumer reads the CometD stream, decodes each event, and writes to Snowflake. Last night the consumer started logging ChangeDataCapture: missing ChangeEventHeader and skipping events. The CDC channel is still publishing, but each message arriving is missing the metadata header your decoder relies on.

What CDC events actually contain

Each Change Data Capture event has two parts: the ChangeEventHeader and the changed-field payload. The header carries the metadata the platform uses to identify and order events: the entity name, the record id, the change type (CREATE, UPDATE, DELETE, UNDELETE), the timestamp, the changed-fields list, and the replay id used for stream resumption.

The payload section contains the actual field values that changed: the new value of Name, the new Industry, the new Phone, and so on.

If your consumer receives a message and can't parse the ChangeEventHeader, one of several things is wrong. The schema used by your decoder might be outdated compared to the platform's current event shape. The transport might be re-encoding the message and stripping fields. The consumer might be subscribing to the wrong channel. Or you might be replaying from an old replay id where the event schema has since been compacted.

The broken example

A Kafka consumer that assumes a fixed event shape:

function handleEvent(message) {
    const payload = JSON.parse(message.value);
    const header = payload.ChangeEventHeader;
    const recordId = header.recordIds[0];
    const changeType = header.changeType;
    upsertToWarehouse(recordId, changeType, payload);
}

The decoder expects ChangeEventHeader to always be at the top of the payload. When the message arrives in a different shape (because the CometD response wraps the event differently than expected, or because the consumer is subscribing to a different channel), header is undefined and the function crashes.

A second shape, an Apex trigger on the standard change event channel:

trigger AccountChangeTrigger on AccountChangeEvent (after insert) {
    for (AccountChangeEvent event : Trigger.new) {
        EventBus.ChangeEventHeader header = event.ChangeEventHeader;
        for (String recordId : header.recordIds) {
            // Process each affected record id.
        }
    }
}

If the trigger runs in an org where CDC isn't enabled for Account, the trigger fires but event.ChangeEventHeader is null. The dereference throws. Or the trigger fires but the record IDs list is empty because CDC was disabled mid-batch.

Three paths to a fix

The likeliest causes in order:

Subscribe to the right CDC channel. Standard CDC channels are named /data/<ObjectName>ChangeEvent for standard objects (/data/AccountChangeEvent, /data/OpportunityChangeEvent) and /data/<ObjectName>__ChangeEvent for custom objects (/data/Renewal__ChangeEvent). If your consumer subscribed to /event/AccountChangeEvent (the platform event channel pattern, not the CDC one), you'll receive a different message shape that doesn't include the same header. Fix the channel.

Enable CDC for the object in Setup. Setup, then Change Data Capture. Move the object from the "Available Entities" list to "Selected Entities". Without this, CDC events for the object aren't published at all. If your consumer subscribed before the object was enabled, the stream is silent rather than malformed; if the object was enabled then disabled, the stream goes silent and replay attempts can return malformed events.

Update the event schema in your decoder. Salesforce CDC events follow the Salesforce schema. If your consumer uses a generated Avro or JSON schema, regenerate it from the org's current describe data. Schemas can drift when new fields are added or when the org upgrades to a new API version that changed event shapes.

The fixed example

A defensive Kafka consumer that handles missing or malformed headers:

function handleEvent(message) {
    let payload;
    try {
        payload = JSON.parse(message.value);
    } catch (err) {
        logError('Malformed JSON payload', { offset: message.offset });
        return;
    }

    const header = payload.ChangeEventHeader;
    if (!header || !Array.isArray(header.recordIds) || header.recordIds.length === 0) {
        logError('Missing or empty ChangeEventHeader', {
            offset: message.offset,
            keys: Object.keys(payload).join(',')
        });
        // Acknowledge and skip; do not crash the consumer.
        return;
    }

    const { entityName, changeType, recordIds, commitTimestamp } = header;
    for (const recordId of recordIds) {
        upsertToWarehouse({
            entityName,
            recordId,
            changeType,
            commitTimestamp,
            fields: payload
        });
    }
}

The same defensiveness in an Apex trigger:

trigger AccountChangeTrigger on AccountChangeEvent (after insert) {
    for (AccountChangeEvent event : Trigger.new) {
        EventBus.ChangeEventHeader header = event.ChangeEventHeader;
        if (header == null || header.recordIds == null || header.recordIds.isEmpty()) {
            System.debug(LoggingLevel.WARN, 'CDC event with missing header; skipping.');
            continue;
        }
        for (String recordId : header.recordIds) {
            AccountChangeHandler.process(recordId, header.changeType, event);
        }
    }
}

The replay id story

CDC streams use a replay id mechanism for resumption. When your consumer disconnects and reconnects, you can resume from the last replay id you processed, which avoids losing or duplicating events.

The replay window is limited. Salesforce retains events for around 72 hours (3 days). If your consumer is offline longer than that, the replay ids you held are stale. Replay attempts from those ids may either return the events with truncated payloads or fail entirely.

When resuming after a long outage:

1. Replay from the oldest available replay id (most consumers expose this as a sentinel value).
2. Detect duplicates on the receiving end using the event's record id and commit timestamp.
3. For records that may have been missed entirely, run a full reconciliation query.

The CometD client libraries usually handle replay-id management for you. If you've rolled your own consumer, make sure your replay-id storage is durable across consumer restarts.

CDC vs Platform Events vs Streaming API

Three Salesforce eventing systems exist; they're not the same:

Change Data Capture publishes events when records change. Channel: /data/<Object>ChangeEvent. Shape: ChangeEventHeader plus changed fields. Purpose: keep external systems in sync with database state.

Platform Events are custom events your app emits explicitly via EventBus.publish. Channel: /event/<Event>__e. Shape: any fields you defined on the event object. Purpose: event-driven integration patterns.

PushTopic Streaming is the legacy mechanism. Channel: /topic/<PushTopic>. Shape: defined by the SOQL of the push topic. Purpose: pre-CDC change feeds. Deprecated for new use cases.

If your consumer expects CDC shape and is subscribed to a Platform Event or PushTopic, the header is structurally different and your decoder fails.

What ChangeEventHeader contains

The header is structured as:

{
  "ChangeEventHeader": {
    "entityName": "Account",
    "recordIds": ["001xx000003DGb1AAAA"],
    "changeType": "UPDATE",
    "changeOrigin": "com/salesforce/api/soap/60.0;client=client_id",
    "transactionKey": "abc-123-def-456",
    "sequenceNumber": 1,
    "commitTimestamp": 1716576000000,
    "commitNumber": 12345678,
    "commitUser": "005xx000001Sv8z",
    "nulledFields": [],
    "diffFields": [],
    "changedFields": ["Industry", "AnnualRevenue"]
  },
  "Industry": "Technology",
  "AnnualRevenue": 50000000
}

changedFields lists which fields are in the payload. Fields not in changedFields were unchanged and aren't included. nulledFields is for fields that were explicitly set to null. diffFields is for fields whose values changed in unusual ways.

Knowing the header structure helps debug "missing field" issues. If your warehouse expected Email but the event doesn't include it, the field probably wasn't changed in that transaction (so CDC didn't publish it).

Polling vs streaming

CDC is streaming by design. Some consumers attempt to "poll" CDC by repeatedly subscribing, reading what's available, and disconnecting. This pattern misses events. The replay window helps but is bounded.

For production replication, run a long-lived streaming consumer. If your platform doesn't support that (a serverless function, for example), use intermediate buffering: a small persistent consumer that writes events to S3 or Kafka, then your serverless workers read from there.

A common architecture:

Salesforce CDC --> CometD subscriber (always-on) --> Kafka --> warehouse loaders

The CometD subscriber handles replay-id management and event acknowledgment. Kafka provides durable buffering. Warehouse loaders consume from Kafka at their own pace.

Test patterns

Two tests cover most CDC integration regressions:

A schema-conformance test that fetches a real CDC event from a sandbox and confirms your decoder accepts it without crashing. Run this against every supported object.

A header-presence test that constructs a fabricated malformed event (header removed, recordIds empty, changeType missing) and confirms your decoder logs the issue and continues rather than crashing.

@isTest
static void changeEventDecoder_handlesMissingHeader() {
    AccountChangeEvent malformed = new AccountChangeEvent();
    // Don't set ChangeEventHeader; simulate the bug.
    
    Test.startTest();
    try {
        AccountChangeHandler.handleSingle(malformed);
        // Should not throw.
        System.assert(true, 'Handler tolerated missing header');
    } catch (NullPointerException npe) {
        System.assert(false, 'Handler should not throw on missing header');
    }
    Test.stopTest();
}

The org-wide CDC limit

Each org has a published events limit based on edition. Enterprise Edition gets around 250,000 published events per 24-hour rolling window. If you exceed it, CDC drops events. The downstream consumer sees gaps but the header isn't missing on the events that did get through; it's just that some events never arrived.

If your warehouse is missing rows but the events you receive look fine, check the org's event publishing usage in Setup, then Limits. A drop in event count vs. a peer-day suggests throttling.

Custom-object CDC quirks

For custom objects, CDC must be enabled per object in Setup. Standard objects have CDC pre-defined for the common shapes (Account, Contact, Opportunity, Case, Lead) and require enabling for others.

If your custom object uses field-level encryption, CDC events for encrypted fields contain the encrypted value (or are masked entirely depending on your encryption policy). The downstream system must be able to handle encrypted payloads or have access to the decryption mechanism.

Filtered subscriptions

You can subscribe to a filtered CDC channel using a ChangeEventConsumer configuration. The filter lets you receive only events matching certain criteria, reducing volume.

Channel: /data/AccountChangeEvent
Filter: changeType IN ('CREATE', 'UPDATE') AND Industry = 'Technology'

If you're not seeing events you expect, double-check the filter. A too-restrictive filter looks like missing events from the consumer's perspective.

Reconciliation patterns

For warehouse replication, a periodic reconciliation query catches gaps:

-- Run nightly on the warehouse:
SELECT salesforce_id, last_modified_date
FROM warehouse.account
WHERE last_modified_date > (NOW() - INTERVAL '24 hours');

-- Compare to:
SELECT Id, LastModifiedDate
FROM Account
WHERE LastModifiedDate > LAST_N_DAYS:1;

Mismatches indicate either dropped CDC events or warehouse processing errors. A reconciliation job that emits metrics tells you quickly when CDC is missing events.

Related errors

CDC-related errors often appear together. A small map:

• Missing ChangeEventHeader (the topic of this page)
• Replay id is too old (the replay window expired; reset and reconcile)
• EventBus.publish failed (a Platform Event publish failed; not CDC, but adjacent)
• Channel not found (subscribed to a channel that doesn't exist or isn't enabled)
• Insufficient capacity (org exceeded its event quota)

Each has a different remediation. The header-missing case is usually about decoder shape; the replay-id case is about consumer state; the capacity case is about org configuration.

Habits that prevent CDC pain

Three habits eliminate most CDC integration issues:

Treat every event as potentially incomplete. Code that depends on a specific field being present will eventually break when that field wasn't changed in the source transaction. Read from the payload defensively, with explicit null checks and sensible defaults. The CDC contract is "fields that changed are included"; it isn't "every field is always present." Code that assumes the latter looks correct in test but fails the first time production stops updating a particular field.

Persist your replay id durably. The cheapest source of CDC pain is a consumer that loses its replay id on restart and starts replaying from beginning, generating duplicates downstream. Use a database write or a persistent file, not in-memory state. Check the storage on restart, and if the stored id is older than 72 hours, fall back to a full reconciliation rather than blindly replaying from the earliest available point.

Build reconciliation into the pipeline. Streaming is "best effort" with retry windows; a reconciliation step turns "best effort" into "verified eventual consistency." A daily query that compares record counts and last-modified-date ranges between Salesforce and your warehouse catches the gaps that streaming alone can't promise to fill, and gives you a way to detect drift before the business team notices.

Further reading from Salesforce

• Change Data Capture Developer Guide
• Salesforce Help: Enable Change Data Capture
• Pub/Sub API: Subscribe to CDC Events
• Platform Events Developer Guide
• Trailhead: Change Data Capture Basics