Task container contract

This page sets out what Flow hands a task's container and what it takes back. It's for task developers who need the exact rules, and for developers whose own dispatcher runs task images. For a walkthrough that builds a task and adds it to a workspace, see Develop a task.

Which task types run in a container?

Seven task types go to a dispatcher. The engine runs every other type itself, with no container.

A dispatcher receives only the types it registered. Flow's Kubernetes dispatcher registers every dispatched type except generic, so a generic task waits in the queue until a dispatcher that registers it is running.

• template and script run the task's image with its command, arguments or script. A task with no image takes Admin › Settings › Tasks › Default Image Path when its task run is created.
• custom takes its runtime from its own parameters: image, command and arguments (each split on newlines), and shellScript. A secret command or arguments arrives as one element.
• uploadartifact and downloadartifact: Flow fills in the transfer links when it hands the task out, and verifies an upload when the task ends. An optional download of a missing artifact ends succeeded with the result found=false, without reaching a dispatcher.
• ai runs the image its revision declares. See How AI tasks run.

When are references filled in?

Flow fills in every reference before a dispatcher sees the task, at two moments:

So $(params.x) in a script, command, argument or environment variable reads the task's own parameter as frozen at admission, and a requeued task gets the same values. A $(workspace.params.x) or $(global.params.x) written straight into one of those four fields is read at the claim, so an edit made since admission shows.

Flow inserts each value byte for byte: quotes, newlines, backslashes and $ included. Shell syntax such as $(date) isn't a Flow reference and stays as written. A reference to something undeclared fails the task before any dispatcher can claim it, with the reason ParamUnresolved. A reference that can't be filled at the claim, because its workspace or global parameter was deleted, fails the task with SpecFillFailed.

What does the container receive?

Flow's Kubernetes dispatcher sets these environment variables. A later row wins on a name clash, so the task's own variables override the rest.

The container also gets:

• /data: a scratch directory of its own;
• the run workspace, when the workflow declares one, at the workflow's mount path (/workspace by default);
• a script: on kube-jobs it's mounted at /scripts/script and runs as a file, so it must start with a shebang such as #!/bin/sh. On tekton it runs as the step's script.

There's no /params directory and no variable holding every parameter as JSON. Pass a large input on the run workspace, with a parameter carrying its path.

How names and values are encoded

• Names: Flow upper-cases each name and replaces any character outside A-Z, a-z, 0-9 and _ with _. So api-url arrives as PARAM_API_URL, and privateKey as PARAM_PRIVATEKEY. PARAM_NAMES maps each variable back to its exact name.
• Clashes: Flow refuses to save two names that fold to the same variable, with PARAM_NAME_COLLISION. The dispatcher repeats the check and fails a task that still has one.
• Values: a string arrives as it is and a null as an empty string. Numbers, booleans, arrays and objects arrive as JSON.
• For-each items: each item also gets PARAM_ITEM, the element (JSON when it's an object or array), and PARAM_INDEX, its position from 0 as text. See Repeat a task for each item.

Secrets arrive as plain environment variables

A secret parameter reaches the container as an ordinary PARAM_<NAME>, in plain text. Flow masks secrets on every read and scrubs the run's secret values from the logs it serves, but there's no per-task Kubernetes Secret.

How does a task report its outcome?

Exit 0 to succeed. Any other exit code fails the task. When a task fails, Flow's Kubernetes dispatcher sends a status message of up to 4096 bytes: the Job's or TaskRun's condition, the container's exit code and termination reason, and the last 20 lines of its log.

Write results to RESULTS_PATH:

Only declared results count. Tekton collects only the results the task declares, and on kube-jobs Flow keeps only the declared names when the task declares any. On kube-jobs, the termination message carries results and nothing else: a short message that isn't a results payload is logged and returns no results. Write errors to the log instead: the status message carries its last 20 lines. Develop a task has a shell snippet that works on both executors, and the Node library @boomerang-io/task-core.

What are the size caps?

A node's retry setting never retries ResultsTooLarge. Move larger data by reference: a path on the run workspace, or an artifact. See Keep files between runs.

Which images does the catalogue use?

Catalogue tasks are released from the boomerang-io/tasks repository on their own version lines, apart from Flow's.

A workflow pins each node's task version when it's saved, so a new revision changes nothing until the workflow is saved again.

The task-flow tasks read their parameters from PARAM_NAMES and PARAM_<NAME>, and write results to RESULTS_PATH, through @boomerang-io/task-core. A dispatcher you write must set those variables to run catalogue images.

How AI tasks run

An ai task runs the image and command its revision declares, prompt or agent. It never runs a script, and a workflow author never builds its container. An in-process dispatcher ignores the image and does the work itself.

Flow's Kubernetes dispatcher runs an ai task only when its image starts with a prefix the operator allows, boomerangio/task-ai by default. Any other image fails the task with ImageNotAllowed before anything is created, and a node's retry setting never retries it. To run an AI worker of your own, your operator adds its prefix: see Flow's Kubernetes dispatcher.

An ai pod also receives, when the operator sets them on the dispatcher:

• OPENAI_BASE_URL and OPENAI_API_KEY, a default endpoint and token. The task's own endpoint and token parameters still arrive as PARAM_ENDPOINT and PARAM_TOKEN, and win inside the image.
• OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS and OTEL_EXPORTER_OTLP_PROTOCOL, the OpenTelemetry exporter settings, with OTEL_SERVICE_NAME set to flow-task-<taskRef> and OTEL_RESOURCE_ATTRIBUTES carrying flow.workflow.run.id, flow.task.run.id and flow.task.name.

A task's own variable of the same name wins. The worker's parameters and results are documented in tasks/ai/README.md in the boomerang-io/tasks repository. For using AI tasks in a workflow, see Use AI tasks.

Join the run's trace

TRACEPARENT is 00-, then 00000000 followed by the run's 24-character id, then -, the last 16 characters of the task run's id, and -01. For example, task run 6ad1bfa6cfefa66e88d5e2e0 of run 6ad1bfa3cfefa66e88d5e24a gets:

00-000000006ad1bfa3cfefa66e88d5e24a-cfefa66e88d5e2e0-01

A task that exports OpenTelemetry spans and reads TRACEPARENT through the W3C propagator lands in its run's trace, beside the run's other tasks. A retry is a new run, so it's a new trace. Flow stores no traces and emits no run or task spans of its own.

Next steps

• Develop a task: build a task and add it in Task Manager.
• Write a dispatcher: run task runs on infrastructure of your own.
• Use secrets: how a secret parameter travels.
• Dispatchers: how Flow hands work to a dispatcher.