HTTP request tasks¶

The optional Orchestra HTTP submodule (orchestra_http) adds an http_request task: a step that calls an external HTTP endpoint and routes on its answer, with no code of your own.

An example¶

A request is registered with another system. The step posts the applicant's name and email to the registry. The registry answers 201 with the new record, or 409 when the request is already there, and the step's flows route the two apart:

flowchart LR
  s([Start]) --> r["Register the request<br/>type: http_request"]
  r -->|status == 201| d([Registered])
  r -->|status == 409| a([Already registered])
  r -->|registration == failed| f["Tell the team<br/>type: user"]

n_register:
  type: http_request
  config:
    method: POST
    url: 'https://registry.example.com/requests'
    body_type: json
    json_body: |
      {
        "applicant": {"name": [orchestra:var:applicant_name], "email": [orchestra:var:applicant_email]},
        "tags": [orchestra:var:tags],
        "source": "orchestra"
      }
    auth: bearer
    credential_key: registry_token
    request_timeout: 10
    payload_variable: status
    response_variable: registry
  features:
    retry:
      max_attempts: 3
      backoff: PT1M
    on_failure:
      policy: outcome
      variable: registration

After the step, status holds 201 and registry holds the decoded answer, so a later step reads registry.id. A registry that cannot be reached is tried three times, a minute apart, and then the step goes on with failed in registration.

Answers and failures¶

A response is an answer, whatever its status. The step writes the status code to its payload variable, as a number, so a flow routes == 404, or >= 200 and < 300 for any success. A redirect is an answer too: the step does not follow it.

Three things are failures instead:

no answer at all: a timeout, a refused connection, a name that does not resolve (a DNS outage explains it as well as a mistyped name);
a 429: the endpoint asking to be called later;
a 5xx: the endpoint failing on its side.

A failure is retryable, so the step's retry policy applies. Once its attempts are used up, the step's If this step fails setting decides: an incident by default, a failed instance, or the failed outcome its flows route, as in the example.

A step that cannot be sent at all fails at once, without a retry: no URL, a host that is not one of its own, a missing credential key, or an answer larger than a step keeps.

Settings¶

method: GET (the default), POST, PUT, PATCH or DELETE.
url: the http or https address to call. Tokens read this instance and branch: https://registry.example.com/requests/[orchestra:var:request_id]. Each token's value is encoded where it goes, so a/b fills one path segment as a%2Fb and x&y=1 one query value. A value cannot add a segment, a parameter or a host.
allowed_hosts: the hosts the step may call besides the one its URL is written with, one per line. Required when a token supplies the host, https://[orchestra:var:region].registry.example.com/, so a variable can only choose among the hosts listed.
headers: one per line, as Name|value, tokens replaced as plain text. A value that would hold a line break fails the step, so a variable cannot add a header of its own.
body_type: none (the default), json, form, multipart or text.
json_body: the JSON document to send, written as it goes out, with tokens where values go. A [orchestra:var:NAME] token is replaced by the variable as JSON: a number stays a number, a list a list, an object an object, and a variable that is not there is null. A dotted name reads into a structured variable, as everywhere (Variables). Any other token, [site:name] or a variable chained into an entity ([orchestra:var:owner:user:display-name]), is replaced by a JSON string. A token goes where a value goes, never inside quotes: the form refuses "Hello [orchestra:var:name]", which would break the document. A literal array holding a colon, ["https://example.com", "10:30"], is not a token and is sent as written. Sent as application/json.
form_fields, for form and multipart: one field per line, as name|value, tokens replaced as plain text. form sends them application/x-www-form-urlencoded, multipart as multipart/form-data, text fields only.
body and content_type: a text body, tokens replaced as plain text, and its content type (text/plain by default).

A Content-Type header line, in any case, wins over the one the body type sets. - auth: none (the default), basic with a username, or bearer. The password or the token is held by the Key module, named in credential_key, so the secret stays out of the workflow's configuration. The workflow depends on that key, so the key cannot be deleted while the step signs in with it. - request_timeout: how many seconds to wait for an answer, 10 by default, at most 120. - payload_variable: the variable the status code is written to. It cannot be the response variable, whose answer the status code would replace, nor a name inside it, such as response.status, which would hide the answer's field of that name. - response_variable: the variable the response body is written to. A body the endpoint says is JSON is decoded, so a flow or a later step reads response.items.0.id into it; any other body is kept as text. A body larger than 256 KB fails the step, rather than being cut: a cut JSON document no longer decodes. A step with no response variable does not mind the size, and a 429 or a 5xx is retried whatever its body. - store_fields_separately, split_objects, split_lists and fields_to_store: whether the answer is stored as one variable or field by field, and which fields; see Storing the answer field by field. - completion_scope: instance (the default) or token, where both variables are written.

Storing the answer field by field¶

By default the whole answer is one variable, the response variable, and a flow reads a part of it by path: in the example above, registry holds the answer and registry.id reads its id. That is enough for flow conditions and tokens. A part becomes a variable of its own, a row the variable list shows and a view filters on, only when the step stores the answer field by field. The Wait for an event step takes the same four settings for an event's data (see Waiting for an event).

Each variable is named after the response variable, a dot, and the field's name, so registry.id names the same value either way, and a flow reads it the same way whichever the step chooses. Take this answer, with response_variable: registry:

{"id": 42, "status": "pending", "applicant": {"name": "Ada"}, "tags": ["a", "b"]}

Settings	Variables written
none (the default)	`registry`: the whole answer
`store_fields_separately`	`registry.id` = `42`, `registry.status` = `"pending"`, `registry.applicant` = `{"name": "Ada"}`, `registry.tags` = `["a", "b"]`
and `split_objects`	`registry.applicant.name` = `"Ada"`, in place of `registry.applicant`
and `split_lists`	`registry.tags.0` = `"a"`, `registry.tags.1` = `"b"`, in place of `registry.tags`

Each split goes one level down: an object inside applicant would stay one variable, registry.applicant.address. With both splits, a list of objects gives registry.items.0.id.

An answer that is a list¶

A list has no field names, only positions, so it is split only when split_lists asks for positions. Each item is then a field, and split_objects splits it in turn. Take [{"id": 1}, {"id": 2}]:

Settings	Variables written
`store_fields_separately`	`registry`: the whole list
and `split_lists`	`registry.0` = `{"id": 1}`, `registry.1` = `{"id": 2}`
and `split_objects` too	`registry.0.id` = `1`, `registry.1.id` = `2`

So with both splits, fields_to_store set to registry.*.id (see below) keeps one variable per entry, registry.0.id, registry.1.id and so on, however many entries the list has, and leaves out every other field of the entries.

An answer that is a single value or text has no fields, and stays whole.

Keeping only some fields¶

fields_to_store keeps only some of the variables the splits wrote, one pattern per line, each named as stored. It chooses among them and never splits anything itself: the splits decide how far the answer is split. With the first answer above, and both splits:

Line	Keeps
`registry.id`	`registry.id`
`registry.*`	each variable one level below `registry`: `registry.id`, `registry.status`; not `registry.applicant.name`, two levels down
`registry.tags.*`	`registry.tags.0`, `registry.tags.1`
`registry.**`	every variable below `registry`, at any level
`registry`	the whole answer as well, beside the variables kept

* stands for any one level, including in the middle: registry.items.*.id keeps each item's id. ** stands for one level or more, and only goes last. The response variable alone keeps the whole answer beside the variables the other lines keep: flows can read any part of it by path, while the fields that need a row of their own get one. Left empty, the list keeps every variable.

The form refuses a line that is neither the response variable alone nor the response variable followed by a dot, a wildcard inside a name, such as registry.it*, and a ** anywhere but last. Such a line fails the step if it reaches it another way, such as configuration written by hand.

What stays from before¶

Nothing deletes a field a later answer no longer has: when the step runs again, in a loop for example, a field it is not given keeps its value from before, and so do the last positions of a longer list. Store the answer whole when that matters. A field whose name would make a variable name longer than 255 characters fails the step, as storage keeps no longer name: store whole an answer keyed by such data, or keep only the fields you need.

What a step may call¶

A step calls the servers its author named, whatever a variable holds. A URL can be built from variables a form filled in, so this is what protects the site from a value chosen to send the request elsewhere:

A value cannot change the host. Each token's value is encoded where it goes into the URL, so it fills a path segment or a query value and nothing else: x@other.example/ stays inside the path.
A host a token supplies is one the author listed. The step calls the host its URL is written with, or one of its allowed hosts; any other fails the step before anything is sent.
Redirects are not followed, so an answer cannot send the step to a server nobody named.

Plain http is allowed, an intranet service often being served over it. Credentials sent over it (basic or bearer) travel unencrypted, so saving a workflow with such a step shows a warning. Use https unless the network between the site and the service is trusted.

Synchronous workflows wait for the answer¶

The call is made inside the advance that runs the step. A workflow running synchronously makes it inside the request that advanced it: the person who submitted the form waits for the endpoint. Call a slow endpoint from a workflow running on the queue (Execution modes), and keep the timeout short.

The example workflow¶

With orchestra_examples enabled too, example_http_lookup looks up the request named by request_id at https://api.example.com/requests/, and routes a 200, a 404, and an endpoint it could not reach to three different ends. Point its URL at an endpoint of your own to try it.