Skip to content

Flows, nodes and messages

Three ideas hold the whole system up. They are worth twenty minutes, because almost everything else follows from them.

A flow is a graph you did not draw

A flow is a set of nodes. Each node declares the messages it needs (requires) and the messages it produces (provides). The graph is whatever those declarations imply:

# node "read"
def process():
    return {"temperature": read_sensor()}

# node "decide"
def process(temperature, setpoint=21.0):
    return {"heat": temperature < setpoint}

decide is downstream of read because it needs temperature and read produces it. Nobody drew a wire.

This is the one structural decision everything else rests on, so it is worth being explicit about the consequences:

  • Fan-in is free. Two nodes providing temperature are two producers of one message. The consumer does not change.
  • A wire cannot be wrong. There is no wire. There is a name that either matches or does not, and the canvas tells you at edit time which it is.
  • Layout is not a document. The canvas computes the arrangement, so a flow has no stored positions to maintain, merge or fight over.
  • Flows stay small. A graph nobody can hand-arrange is one worth keeping small — which is the intent. Several atomic flows that name each other beat one flow with sixty nodes in it.

Message names are namespaced

Inside flow house, a message named temperature is really house.temperature. A bare name is qualified with its own flow; a dotted name is used as written. That is how two flows share a value:

# in flow `dashboard`, reading a message that flow `house` produces
def process(house_temperature):     # port bound to "house.temperature"
    ...

The canvas draws messages arriving from another flow as labelled endpoints, so you can see where they come from without opening the other flow.

A node is a function with declared ports

Most nodes are Function nodes: a Python file defining process(...). Its arguments are its input ports by name; its return value is a dict keyed by output ports.

def process(reading, unit="C"):
    return {"shown": reading if unit == "C" else reading * 1.8 + 32}

reading is a port. unit is a setting — a constant of this node's code, typed into its panel and stored with the flow. Both arrive as arguments, which is why a setting may not share a name with a port. See Where a node's values come from.

The rest of the node types are the ones that would be tedious or unsafe to write yourself: MQTT, HTTP, InfluxDB, schedules, switches, notifications. Each one is configured by filling in a form the editor generates from its parameter schema, so they all behave the same way. The full list is in Node types.

Ports are typed

A port declares a dtype: float, int, str, bool, json, record, list, series or artifact. Every value that passes through is checked against it.

Types are not decoration. They are what lets the dashboard editor offer you only the messages a gauge can actually draw, and what lets the canvas refuse a binding before anything runs. See Payload types.

Everything on the wire is JSON. Bytes — a checkpoint, an image, a model — travel as an artifact: the bytes go to a content-addressed store and the message carries a small reference to them.

Nodes are pure

A node is called with the values of the messages it declares and returns the values of the messages it provides. There is no context object, no global store, no handle to reach for.

That is deliberate: a node with hidden state cannot run twice in parallel, cannot be replayed, and cannot be moved to another machine. Plenty of real automations do need to remember something, and there is a specific way to say so — see Keeping state in a flow.

Two shapes of flow

Set mode on the flow:

live (default) batch
Runs continuously once per run, on request
Started by subscriptions, schedules, webhooks POST /runs/flows/{name}
Ends never when the graph drains
Keeps the last value of each message a run record: params, result, metrics, artifacts
Is a thermostat, an ETL job on a cron an experiment, a CI-style job

A batch flow is built and validated like any other, appears on the same canvas and is type-checked the same way. It is simply never activated: no subscriptions, no schedules, no webhooks. See Runs: pipelines that finish.

Editing is separate from running

Every flow has a published version and, while you are working, a draft.

  • Saving writes the draft. The engine keeps running the published version.
  • Publishing promotes the draft. The engine reloads and picks it up.
  • Discarding throws the draft away.

The store is a git repository — flow.json for the structure, nodes/*.py for the code — and each save is a commit. So a flow's history is readable with ordinary git tooling, and copying a flow between installations is copying a directory.

Saving carries the version you last saw. If someone else saved in between, you get a 409 instead of quietly overwriting their work.

What can be wrong, and when you find out

The canvas validates continuously and names problems on the nodes they belong to:

Issue What it means
unconnected_input a port needs a message nothing in reach provides
missing_initial_value the message exists but has never held a value, and nothing will give it one
cycle A waits for B and B waits for A — nothing could ever start
self_loop_needs_initial a node reads a message it also writes, with no starting value
node_error the node's code did not load: a syntax error, a missing import
unauthenticated_hook advisory — a webhook with no shared secret is open to anyone

A flow with any of these except the advisory one does not run. The health summary on Home counts them, so "why is nothing happening?" has an answer that does not involve reading logs.

What happens at runtime

  • A flow can be started and stopped. Stopped means its subscriptions and schedules are torn down.
  • A flow can be paused and stepped. Paused holds messages instead of running them; step releases exactly one. This is how you test something before it moves a relay.
  • A failing node does not take the flow down. It reports an error, keeps its last error visible after it recovers, and can fire an alert.
  • A flow whose background tasks keep crashing is quarantined. The engine stops restarting them and says so, rather than spinning. Publishing a change gives it another chance.

Values that arrive from outside

Some messages are not computed by any node: a dashboard control writes them, the API publishes them, a batch run passes them in. Declare those as the flow's inputs, with the value they start from:

{"inputs": [{"spec": {"name": "setpoint", "dtype": "float"}, "initial": 21.0}]}

Without that, the node reading setpoint waits for something nothing provides, and the canvas says so. With it, the flow starts at 21.0 and whatever writes the message afterwards takes over.

Where to next