Troubleshooting

Orchestra advances steps independently and in parallel, so a few rare states can arise that most runs never hit. This page lists the symptoms and what to do about each.

An instance stuck at a join

Symptom: an instance stays RUNNING with an incident saying a join can no longer fire, or, until the cron run that halts it on one, with one or more WAITING tokens at a join and nothing moving.

A branch a join waits for can be canceled or end elsewhere. For example, an operator resolves a dead-lettered branch with "Cancel" (see Incidents). The join then decides again at once:

• a wait_all join halts on an incident: a path drawn into it will not deliver;
• a matching, threshold or quorum join stops waiting for it.

A join can still be left waiting when a branch goes without the engine seeing it, for example a token changed straight in the database. The branches already at the join then have nothing to fire them. The engine looks for this case: an instance whose live tokens are all WAITING with no deadline. It looks when a cancellation strands an instance, and on each cron run. What it does then:

• for a join whose node has a usable join_timeout, it arms the deadline from the instance's definition, so the sweep releases the join;
• for any other join, one branch waiting there halts on an incident, so the instance shows up among the incidents instead of looking healthy.

A note on names: the WAITING token state is join synchronization only. A wait node, and every external interaction (a webform or capability link, an external task handler), parks its token as PARKED, not WAITING. An instance idling on external input is therefore not a stranded join, and is not flagged as one.

Fix: resolve the incident. Fire the join goes on with the branches that arrived. To give up on the missing branch after a delay instead, give the join a timeout and Retry:

1. Give the join's node a join_timeout, either by editing the workflow (which reaches the stuck instance only if it pins no version, so only if it started while the mode was Off - switching the mode off now does not unpin a run already going) or by migrating the instance to a version whose join has one (see Versioning).
2. Retry the incident: the branch waits at the join again, this time with a deadline, and the timeout sweep fires the join once it passes.

If the run is not worth finishing, fail the instance instead.

A run that ended right after one step, looking successful

Symptom: an instance is COMPLETED immediately after a step, with none of the steps that should follow it, and the person who submitted the form saw a normal confirmation.

The token reached a node that has outgoing flows and took none of them: every condition evaluated false, so no successor was produced, the branch ended and the instance completed. A typo in a condition, a variable no provider wrote, a renamed key, or an outcome value no flow matches all take that shape.

The engine now reports it: the branch is dead-lettered to an incident naming the node, the conditions it evaluated and the value each one saw, and the instance stays RUNNING until an operator resolves it (see Incidents).

Fix: read the incident's message. It names the condition and what it actually read, which is normally the whole diagnosis:

1. If the value is NULL, the variable the condition names was never written under that name: fix the condition, or the task that should write it.
2. If the value is present but matches no flow, add the flow that routes it (or a catch-all flow to an end node).
3. Then Resume the incident with the corrected variable, or Retry it to re-run the node. Cancel branch abandons it.

If a workflow ends branches on guarded flows on purpose, model that with a catch-all flow to an end node rather than relying on the dead end. The site-wide escape hatch is When a branch takes none of its outgoing flows, under When a step cannot go on on the Orchestra settings page.

A human step whose task is in nobody's inbox

Symptom: an instance stays RUNNING on a human step whose task appears in no inbox and no pending-actions list. Nobody can claim it, and nothing moves.

A task is offered to the audience its step resolved when the token parked, and those candidate tokens are then frozen on it. Nothing asks again, so the pool can empty afterwards: the last member leaves the role the task is pooled to, or every account it names is blocked or deleted. The task is still open and the run is still running, so nothing about either reads as wrong, and no query on the run finds it. What changed happened to a role, or to an account that never held the task.

Each cron run asks the audiences whether their own tokens still name anybody, and halts a step whose every candidate came back with nobody. The run then carries an incident naming the node and the audience that emptied, which is where the diagnosis is (see Incidents), and the halt takes the task nobody could see with it. One case is not reported and has to be spotted by hand: a token minted by a custom assignment plugin that cannot enumerate its membership answers nothing, and since a step is halted only when every candidate has emptied, one such token keeps its own step out of the sweep.

Fix: put somebody back in the audience, then resolve the incident.

1. Read the incident's message. It names the node and the audience that emptied, by label: a role reads as its own label, a named account as its display name, and an account that has been deleted as the user:5 token nothing is left to read a name from.
2. Give that role a member, or edit the node's assignment to name accounts that can still sign in.
3. Retry the incident, which re-runs the node and resolves the audience afresh against the accounts as they now are. For an audience read from a process variable, Resume writes the corrected variable and retries in one action.

A timeout or an escalation timer that never fires

Symptom: a task waits past its deadline, or a reminder ladder never reminds. The task is PARKED as normal, the timer tokens beside it are PARKED too, and the cron sweep runs without touching any of them.

A deadline is a moment stamped on the token, and the sweep fires what is due. Where the moment could not be worked out there is nothing stamped, and a token with no deadline is indistinguishable from one deliberately waiting to be resumed - which is why the ladder looks armed. Three configurations produce it, and the engine now names each in the log:

• A timer whose after offset is not a duration. Timer @index on node %node has offset %after, which is not a duration names the node and the entry in the ladder. The authoring form refuses such an offset, so this arrives by configuration import, where the schema types the offset as a plain string.
• A recurring timer whose offset stopped being measurable while it was running, because the node was edited under the alarm. The fire before that edit was silently its last: it re-armed with no deadline and will not fire again.
• A relative timeout whose duration names a process variable that holds nothing usable. Timeout duration %duration resolves to no usable duration names it. The site-wide default is deliberately not used there, because firing on a window nobody configured would be worse than not firing.
• The site-wide Default timeout setting itself not being a duration, which is the widest of the four: every relative timeout on the site then parks without a deadline. The site default timeout %duration is not a duration names it. Empty there is a real answer and says nothing, because a site that names no default simply leaves a node that names none without a timeout.

The first message comes in two forms, and the difference is worth reading. Where another rung of the same ladder still has a usable offset, only the broken rung is lost. Where none has, the message says nothing will move this token on its own. A timeout written beside a ladder is no consolation there: a node uses either a single timeout or timers, never both, so one that declares timers never arms its own, and the ladder is the whole of what could have come due.

None of this strands a run. A parked token can always be resumed or canceled by an operator (see Incidents for the actions), which is why these are logged rather than raised as incidents: an incident means a branch that cannot proceed, and this one can.

Fix: correct the offset or the duration on the node, or the site setting, then re-park the task so the corrected value is read. A recurring timer asks again on its next window, so a ladder that still has a window left recovers on its own once the configuration is right; one whose window is gone needs the task re-parked (an operator Retry on the step, or a migration to a corrected version).

An instance that looks finished but still shows running

Symptom: an instance briefly shows RUNNING with no live token left, right after its last parallel branches ended.

When two parallel branches reach their end at the very same moment under separate cron workers, each can commit its own branch without yet seeing the other's, so neither marks the instance complete. This resolves itself: a cron reconciliation pass completes any running instance that has no live token and no open incident, so no action is needed. It is listed here only so a momentarily "finished but still running" instance is not mistaken for a real problem.