Replay from the failed step, not from record one

Every integration platform can move a record from A to B. The difference shows up the morning after something broke overnight. A payout file with half a million rows failed at row 480,000 because the ERP rate-limited the connection at two in the morning. Now someone has to decide: rerun the whole job and hope the first 480,000 rows are safe to write twice, or hand-pick the tail and load it by hand.

Both options cost a day. The first one also risks doubled journal entries, which is a far worse problem than a late one.

Resuming from the failed step sounds like a feature toggle. It is closer to an architecture decision, and it has three parts.

• Checkpoints inside the run. Bulk steps process records in chunks and record where each chunk ended. A restart reads the last checkpoint and continues from there, so the 480,000 rows that already landed stay untouched.
• An event-sourced run history. Every step records what it received and what it produced. Replay does not re-fetch the source and hope it looks the same; it continues from the recorded inputs, so a replay the next morning sees the data from the night before. Failed runs keep that data for a set number of hours, so replay is a same-day or next-day tool.
• Idempotent writes where the target supports them. For connectors that accept an idempotency key, such as Stripe, turn on Idempotency key for the step, and a retried step sends the same key, so it never double charges or double books. Where a target has no such key, the platform stays cooperative rather than promising exactly-once delivery it cannot deliver.

Neblex Integration Fabric was built around those three. Bulk data processing covers the chunking and checkpoints, and workflows covers the run history and replay.

The Stripe to NetSuite integration is a good case because the volumes are real and the target is strict.

1. 1
   Trigger. Stripe sends a payout paid event to a Webhook trigger that Neblex registered with Stripe. The flow starts with the payout already in hand.
2. 2
   Fetch. The flow pages through the balance transactions behind that payout.
3. 3
   Transform. Each chunk becomes journal lines with the fee, the net, and the currency conversion split out.
4. 4
   Write. NetSuite receives the journal lines in chunks, and each chunk is recorded as it lands. If the connection is throttled, Retry on failure tries again with backoff. If it fails for good, the run stops with the chunk recorded.
5. 5
   Replay. Someone opens the run, sees chunk 96 of 100 failed, fixes the cause, and replays. The step continues from chunk 96; chunks 1 to 95 are not sent again.

The finance close still happens on time, and nobody reconciles the same payout twice.

The practical change is not speed. It is that a failed integration becomes a bounded task with a known size, instead of an investigation. The run history says which step failed and why, and the fix is to replay from that step.

That is also why we treat the audit trail as the product, not a report on the product. See what we believe for the longer version.