Write a dispatcher

A dispatcher is any process that claims task runs from Flow, runs them and reports how they ended. This page is for developers who write one with the dispatcher SDK. It covers the dependency, the handler, the shapes a dispatcher takes, and testing. The protocol's rules, from claims to compatibility, are in the Dispatcher protocol reference. To deploy the dispatcher Flow ships, see Flow's Kubernetes dispatcher. For how dispatchers fit into Flow, see Dispatchers.

Before you start

• Java 25 or later. The SDK is a Java library. It depends on spring-web (Spring Framework 7) and Jackson 3, and nothing else: no Spring Boot and no Kubernetes client.
• A GitHub token with read:packages, to read the SDK from GitHub Packages.
• A dispatcher token: a global Flow token with an actor kind. Declare one in core's configuration, where the actor kind defaults to SERVICE, or create one in Admin › Tokens with Actor set to Service or Agent. Flow refuses a token with no actor kind. See Give the dispatcher its token.
• Flow's address as your dispatcher reaches it, such as http://flow:7700.

Which shape does your dispatcher need?

The protocol is the boundary, not the process. A dispatcher runs as its own process only where a boundary separates it from Flow's engine: credentials the engine must not hold, a different network zone, isolation of untrusted content, or placement on infrastructure the engine can't reach. Without one, the dispatcher runs inside another process.

Flow's Kubernetes dispatcher is the first shape, built on the same SDK.

Add the SDK

Every product tag publishes the SDK to GitHub Packages as io.boomerang:dispatcher-sdk, at the product's version. Use the version of the Flow you run against:

<repository>
  <id>boomerang-flow</id>
  <url>https://maven.pkg.github.com/boomerang-io/flow</url>
</repository>
 
<dependency>
  <groupId>io.boomerang</groupId>
  <artifactId>dispatcher-sdk</artifactId>
  <version>5.0.0-beta.39</version>
</dependency>

Maven sends your GitHub token from the <server> entry in ~/.m2/settings.xml whose id is boomerang-flow. SLF4J and logback are optional: add them only to capture task logs.

Write the handler

Implement TaskHandler and give it to Dispatcher.builder(). This is a complete dispatcher:

TaskHandler handler =
    (task, context) -> {
      // task.getSpec() has the image, command, script, env; task.getParams() the resolved params.
      String output = runSomewhere(task);
      context.uploadLog(output);                       // optional: the engine stores and serves it
      return TaskResult.of(List.of(new RunResult("exitCode", "0")));
    };
 
try (Dispatcher dispatcher =
    Dispatcher.builder()
        .engine("http://flow:7700", dispatcherToken)   // a Flow machine token with an actor kind
        .name("my-dispatcher")                          // with the host, its identity across restarts
        .taskTypes(List.of("template", "custom", "script"))
        .tasks(handler, () -> 10)                       // at most 10 in flight; 0 is no cap
        .build()) {
  dispatcher.start();                                   // returns at once; registers, then polls
  waitForShutdown();
}                                                       // close() drains for 10 s, then stops

The SDK tells Flow a task is starting before it calls run, and doesn't call it when Flow says not to. It reports the end from what run returns or throws:

StatusReason names every reason a dispatcher sends. Which end reasons requeue a task? says what Flow does with each.

What the handler receives

A TaskRun is the order. These are the parts a handler uses:

A secret parameter's value is plain text on the order, so keep it out of your own logs. A for-each item is an ordinary task run with two more parameters, item and index. To run catalogue images, give the container the variables in What does the container receive?

Flow times a task out itself at its deadline and sends a terminate order. A dispatcher may also enforce timeout on its runtime: Flow's Kubernetes dispatcher does, and reports DeadlineExceeded.

Test your dispatcher

To test a handler, call it with a TaskRun and new TaskContext(task, "d-1", null). To test the client, bind a MockRestServiceServer to the RestClient you pass to DispatcherClient.

The other shapes

A one-shot worker in a warm pool

A microVM or sandbox that takes one task and exits. Give it capacity 1 and stop after the first task:

CountDownLatch done = new CountDownLatch(1);
Dispatcher dispatcher =
    Dispatcher.builder()
        .engine(url, token).name("sandbox").host(sandboxId)
        .taskTypes(List.of("script"))
        .tasks((task, context) -> { try { return run(task, context); } finally { done.countDown(); } }, () -> 1)
        .build();
dispatcher.start();
done.await();
dispatcher.stop();   // the end has been reported by now; the lease ends with the process

Each poll asks for at most one task, so a worker never claims more than it can run. Use a stable name and let host vary. Each worker leaves a registration behind until Flow removes it, 7 days after its last connection. A pool controller decides how many workers to keep warm, but Flow has no queue-depth read to size the pool from.

In-process, inside a host product

The host product both submits runs and runs their tasks. Build the dispatcher as a bean in the host's process, with the host's RestClient so its proxy, certificates and timeouts apply. Capture the handler's logs as the task log, since there's no route for Flow to read them live, and hand it the host's ObservationRegistry so each task's spans join the run's trace:

TaskLogCapture taskLogs = new TaskLogCapture();            // 64 KB head + 64 KB tail per task
TaskLogAppender.attach(taskLogs);                          // logback; see "Logs" in the protocol reference for log4j2
Dispatcher.builder()
    .client(new DispatcherClient(hostRestClient, engineUrl, token))
    .name("in-process").taskTypes(List.of("ai"))
    .tasks(handler, () -> 4)
    .captureTaskLogs(taskLogs)
    .observationRegistry(observationRegistry)
    .build();

There's no workflows(...) and no logs(...): an in-process dispatcher needs neither storage nor a live log route. The token goes on engine calls only. See Embed Flow.

Next steps

• Dispatcher protocol reference: the rules for claims, ends, terminate orders, leases, logs, storage, traces and compatibility.
• Task container contract: what a task's container receives and returns.
• Embed Flow: run Flow inside a host product, with an in-process dispatcher.
• Flow's Kubernetes dispatcher: deploy and configure the dispatcher Flow ships.
• Machine tokens: declare and rotate the dispatcher's token.