Troubleshooting

Orchestra advances steps independently and in parallel, so a few rare states can arise that most runs never hit. This page lists the symptoms and what to do about each.

An instance stuck at a join

Symptom: an instance stays RUNNING with one or more WAITING tokens at a join, and nothing ever moves.

A synchronizing join (wait_all, matching, quorum, threshold) with no timeout waits for the branches it expects. If a branch it is still waiting for is canceled or fails out of band, for example an operator resolving a dead-lettered branch with "Cancel" (see Incidents), that arc never arrives, and with no deadline the siblings already parked at the join have nothing to fire them. Each cron run scans for exactly this shape (an instance whose live tokens are all WAITING with no deadline): when the stuck join is a timeout join it arms the deadline from the instance's definition so the next sweep releases it, and when it is not a timeout join nothing can be armed, so the instance stays stuck for an operator to resolve. The engine logs that unrecoverable case once, when the cancellation strands it, rather than on every cron run.

A note on names: the WAITING token state is join synchronization only. A wait node, and every external interaction (a webform or capability link, an external task handler), parks its token as PARKED, not WAITING. An instance idling on external input is therefore not a stranded join, and is not flagged as one.

Fix: give the stuck join a timeout and let the cron sweep release it.

1. Make the join a timeout join, either by editing the workflow (with versioning off, the change reaches the running instance) or by migrating the instance to a version whose join has one (see Versioning).

The cron sweep then arms the waiting tokens on its next run and fires the join with whatever arrived on the run after, cancelling the branch that never came, so the instance completes. To act immediately rather than wait for cron, call WorkflowEngine::armJoinTimeouts($instance). If no timeout is wanted, cancel the instance instead.

A run that ended right after one step, looking successful

Symptom: an instance is COMPLETED (or, before this was reported, was completed) immediately after a step, with none of the steps that should follow it, and the person who submitted the form saw a normal confirmation.

The token reached a node that has outgoing flows and took none of them: every condition evaluated false, so no successor was produced, the branch ended and the instance completed. A typo in a condition, a variable no provider wrote, a renamed key, or an outcome value no flow matches all take that shape.

The engine now reports it: the branch is dead-lettered to an incident naming the node, the conditions it evaluated and the value each one saw, and the instance stays RUNNING until an operator resolves it (see Incidents).

Fix: read the incident's message. It names the condition and what it actually read, which is normally the whole diagnosis:

1. If the value is NULL, the variable the condition names was never written under that name: fix the condition, or the task that should write it.
2. If the value is present but matches no flow, add the flow that routes it (or a catch-all flow to an end node).
3. Then Resume the incident with the corrected variable, or Retry it to re-run the node. Cancel branch abandons it.

If a workflow ends branches on guarded flows on purpose, model that with a catch-all flow to an end node rather than relying on the dead end; the on_dead_end: complete setting is the site-wide escape hatch.

An instance that looks finished but still shows running

Symptom: an instance briefly shows RUNNING with no live token left, right after its last parallel branches ended.

When two parallel branches reach their end at the very same moment under separate cron workers, each can commit its own branch without yet seeing the other's, so neither marks the instance complete. This resolves itself: a cron reconciliation pass completes any running instance that has no live token and no open incident, so no action is needed. It is listed here only so a momentarily "finished but still running" instance is not mistaken for a real problem.