Replay from the failed step, not from record one

Every integration platform can move a record from A to B. The difference shows up on the Monday after something broke over the weekend. A payout file with half a million rows failed at row 480,000 because the ERP rate-limited the connection at two in the morning. Now someone has to decide: rerun the whole job and hope the first 480,000 rows are safe to write twice, or hand-pick the tail and load it by hand.

Both options cost a day. The first one also risks doubled journal entries, which is a far worse problem than a late one.

Resuming from the failed step sounds like a feature toggle. It is closer to an architecture decision, and it has three parts.

• Checkpoints inside the run. Bulk steps process records in chunks and record where each chunk ended. A restart reads the last checkpoint and continues from there, so the 480,000 rows that already landed stay untouched.
• An event-sourced run history. Every step writes what it received and what it produced. Replay does not re-fetch the source and hope it looks the same; it replays the recorded inputs, so a Tuesday replay of a Saturday run sees Saturday's data.
• Idempotent writes where the target supports them. Connectors that accept an idempotency key, such as Stripe, get one on every write, so a retried step never double charges or double books. Where a target has no such key, the platform stays cooperative rather than promising exactly-once delivery it cannot deliver.

Neblex Integration Fabric was built around those three. Bulk data processing covers the chunking and checkpoints, and workflows covers the run history and replay.

The Stripe to NetSuite integration is a good case because the volumes are real and the target is strict.

1. 1Trigger. Stripe emits a payout paid event by webhook. The flow starts with the payout already in hand.
2. 2Fetch. The flow pages through the balance transactions behind that payout, in chunks, and checkpoints after each chunk.
3. 3Transform. Each chunk becomes journal lines with the fee, the net, and the currency conversion split out.
4. 4Write. NetSuite receives the journal entry. If the connection is throttled, the step retries with backoff. If it fails for good, the run stops with the chunk number recorded.
5. 5Replay. Someone opens the run, sees chunk 96 of 100 failed, and replays from there. Chunks 1 to 95 are not touched.

The finance close still happens on time, and nobody reconciles the same payout twice.

The practical change is not speed. It is that a failed integration becomes a bounded task with a known size, instead of an investigation. The run log says which step failed, on which record, with which input, and the fix is to replay one step.

That is also why we treat the audit trail as the product, not a report on the product. See what we believe for the longer version.