Worker clusters

A cluster lets several Worker hosts share the same flows. Each run is placed on one eligible member. When a member disconnects, the work it had not started yet goes to the other members. A run it had already started is never run again on another member, because it may already have written to your systems: it ends with an unknown outcome when that member restarts or stays away.

1. 1
   Create the cluster. Click New cluster: name, description, Placement policy (Least loaded, Weighted fair, Locality first), Minimum worker version, Required locality, Resource class (General, Input/output, CPU, Memory, GPU) and Member port (7477 unless your network needs another). Save cluster.
2. 2
   Enroll each host as a member. When enrolling a Worker choose the cluster as its execution target. Every host gets its own enrollment code and identity, and you approve each machine.
3. 3
   Check eligibility. Open control center: a member receives work only when it is online, active, current and meets the cluster's version and placement requirements.
4. 4
   Point flows at the cluster. In the flow builder choose the cluster as the run target.

The control center

The control center shows members, jobs versus capacity, queue depth and pending acknowledgements. Each member shows its state: Online, Running, not connected to the platform (it still works and another member vouches for it), Offline or Dormant, with its version. For the cluster it shows:

• Reaches N of M other members for each member. When members cannot reach each other, it shows the advice to allow HTTPS on the member port between the cluster's machines, with the addresses each member reports.
• The member list revision each member holds. Still on roster revision N means changes you made in the app have not reached that member yet.
• Not run while members were isolated: scheduled runs that came due while the member running them could not reach Neblex and that have not run. The flow's missed-run setting decides whether one runs later; start the flow by hand if a missed run matters.

Operations: Pause (Cancel active runs or Let active runs finish), Resume cluster, Drain cluster (no new work, current work finishes), Maintenance / Reactivate, each with an optional change note. Deployed flow controls pause or resume one flow across the cluster. Rotate security key replaces the key that signs the cluster's member list; members take the new one on their own and keep working. Delete cluster is in the danger zone.

Members that reach different systems

Members of a cluster can sit in different networks and reach different systems. Connections on each Worker shows, for every connection the cluster's flows use, which members can use it. Check On Every Worker tests it on each member. A member that cannot use a connection gets no work that needs it and is checked again on its own. If no member can use every connection a flow needs, its runs wait with the reason No On-Prem Worker in the cluster can use all the connections this flow needs.

The member port

Members of a cluster keep an encrypted connection to each other over your network, on the cluster's Member port (7477 unless you choose another in the cluster settings). Allow HTTPS on that port between the cluster's machines, and only between them. Neblex changes no firewall settings. When you change the port, the members switch to it by themselves; a member that cannot open it shows why on the Workers page.

Cluster control

The Cluster control panel shows whether cluster control is set up (see Offline controls), its key generation, and a new control key waiting for your approval. A new key made on a member takes effect only when you approve it here: compare the fingerprint with the one the member's control page shows, then choose Approve key, or Discard key to keep the current key. After a member or an operator is removed, replace the control key on a member's control page.