Capability

Data sync that catches what gets deleted, too

Change data capture with cursor checkpoints, delete detection via content hashing, and schema-drift alerts plus data assertions, keeping operational systems continuously aligned without re-scanning entire tables on every run.

Inserts, updates, deletes
Detects
Cursors and content hashing
Method
Continuous, near real-time
Cadence
Schema drift and data assertions
Safety

The problem

Systems drift out of sync quietly. A record updates in one place and never makes it to the other. A row gets deleted and, because there is nothing left to poll, no downstream system ever finds out. This happens continuously, not once, so a sync has to run on a short interval and know exactly what changed since last time.

Most change-data-capture tooling handles inserts and updates well and treats deletes as an afterthought, exactly the class of bug that shows up as a customer complaint three weeks later, not a failed test.

How data sync and change data capture work on Neblex

Data sync in Neblex Integration Fabric is change data capture (CDC) for operational systems: it runs continuously on an interval and detects inserts, updates, and deletes at the source, not just the changes that happen to leave a trace.

Cursor checkpoints track exactly where each sync left off, so a run picks up only what changed instead of re-scanning entire tables. Delete detection uses content hashing to notice when a record disappears from the source even though nothing was left behind to flag it. For databases, native log capture is available too: Postgres logical replication, SQL Server change tracking, and MySQL binlog, each emitting inserts, updates, and true deletes in commit order.

Schema-drift detection watches the shape of your data itself, flagging a renamed column or a changed type before it silently breaks a mapping three steps downstream.

Data assertions watch the values. Row-count bounds, required fields, numeric ranges, and any condition you write are checked on each batch of changes before it is written. A failed check stops the run with the offending record named, or routes the record onward with the reason to a branch that alerts or parks it for review, so a sync that would blank out or remove an unexpected share of the target does not go through unnoticed.

For a one-time migration or the first full load, that is bulk data processing. Data sync is the version that runs continuously afterward to keep the two systems in agreement.

Conflicts, deltas, and data quality

When a flow runs in each direction, the question is what happens when a record changed on both sides between runs. You resolve it in the sync flow, not with one global switch. A Merge by key step combines the two versions field by field with later fields winning, a branch can hand the conflict to a person through a Wait for form task, or you map a given field in one direction only so one system is its source of truth. The choice is made per field, because a mapping only ever writes the destination fields you choose.

The same inline data quality that a batch load uses runs on each change too: a record is validated before it is written, and a malformed change is routed onward with the reason instead of stalling the sync. A Remove duplicates step on the destination key stops a redelivered or double-detected change from writing twice. See bulk data processing for the full data-quality toolset.

How you build it in Neblex

Flows are built three ways, and you can move between them on the same flow:

  • Natural language. Describe the flow to Matt, the Neblex AI Assistant built into the builder. Matt drafts the canvas, maps fields, and explains steps. It runs on the platform's builder model by default, or on a model you connect for the building experience only. That setting never supplies a model or credential to deployed agents, their tools, knowledge search, apps, or flows.
  • Native builder. Assemble steps, branches, and connectors on the visual canvas.
  • MCP. Create, change, publish, and run flows programmatically, so an external client or agent can build on Neblex directly, with consent scopes for operating, authoring, and publishing.

Agents, forms, apps, data tables, and connections have their own builders and the same MCP surface.

Data sync and change data capture capabilities

  • Change data capture with cursor checkpoints: only what changed, no full table re-scan
  • Delete detection via content hashing, for the deletes most CDC and polling-based sync miss
  • Native log capture for Postgres, SQL Server, and MySQL when the source is a database
  • Schema-drift detection that flags a source change before it breaks a downstream mapping
  • Data assertions on each run: row-count bounds, required fields, numeric ranges, and per-record conditions that stop the run or route the record onward before a bad or destructive change is written
  • Runs on the schedule you set, from every few minutes upward
  • Two-way alignment by running a flow in each direction, not just one-way replication into a warehouse
  • Conflict handling for two-way alignment: field-level merge with later fields winning, route to a person, or a per-field source of truth
  • Each change validated before write and, if malformed, routed onward with its reason without stalling the sync
  • Every sync run recorded in event-sourced history, replayable from the exact step that broke
  • Runs on on-prem workers when the source database cannot leave your network

Worked example

A real sync teams run every few minutes, with the actual step-level detail.

Salesforce and NetSuite customers kept aligned, deletes included

Account changes in Salesforce flow to NetSuite every 5 minutes. A deleted or merged account in Salesforce is detected by hash and deactivated in NetSuite, instead of lingering as a stale record.

  1. 1Read: every 5 minutes the flow pulls Salesforce Accounts modified since the last cursor value.
  2. 2Hash: a content hash of each synced account is stored, so the next run can tell which previously seen accounts no longer appear.
  3. 3Upsert: changed accounts are upserted into NetSuite, keyed on the Salesforce ID stored in a custom field.
  4. 4Detect deletes: accounts present last run but missing this run are flagged as deleted or merged.
  5. 5Deactivate: flagged customers are set to inactive in NetSuite, never hard-deleted, with a note linking the Salesforce record.
  6. 6Flag drift: if the Salesforce fields change shape, the run flags the added and removed fields for review and acknowledgment rather than writing bad data.

Frequently asked questions

How is this different from log-based CDC?

Neblex does both. For databases it can read the change log directly: Postgres logical replication, SQL Server change tracking, and MySQL binlog. For everything else, including SaaS APIs that have no log, it uses cursor-based change detection plus content hashing, which specifically catches the deletes that leave nothing to poll.

How is this different from bulk data processing?

Bulk data processing moves a finite batch: a migration, a backfill, a nightly load, a one-time bulk update. Data sync runs continuously to keep two operational systems in agreement, and its hard problem is detecting what changed since the last run, especially deletes. Use bulk to move a pile of data once or on a schedule, use data sync to keep systems aligned after that.

What happens when a record is changed on both sides between syncs?

You decide it in the sync flow, and you can decide it per field. A Merge by key step merges the two versions field by field with later fields winning. A branch can hand the conflict to a person as a task. Or you map a field in one direction only, so one system is its source of truth. There is no single global conflict policy you have to accept.

Does this only sync into a data warehouse?

No. Data sync and change data capture is built for keeping operational systems aligned, CRM to ERP, ERP to a data table, one SaaS app to another, not just for feeding an analytics warehouse. Two-way alignment is a flow in each direction, with the conflict handling described above.

What happens when a source schema changes unexpectedly?

Schema-drift detection flags the change so an admin can review and adjust the mapping, rather than letting a renamed column or a changed data type silently corrupt data downstream.

How do we catch a bad sync before it writes?

Data assertions run on every batch of changes before they are written: row-count bounds, required fields, numeric ranges, and per-record conditions you configure. A failed check stops the run with the record named, or routes the record onward with its reason to a branch that alerts or parks it, so a run that would delete or blank out an unexpected share of the target does not go through unnoticed.

Does every sync re-scan the whole table?

No. Cursor checkpoints track exactly where the last run left off, so syncs pick up incremental changes instead of scanning entire tables every time.

See it on your data

Bring one real integration

We will build it with you against your own systems, with the run log open.