Skip to content

Long-Lived Assistants

orqest.assistants runs durable, named assistants: each has an address, persistent state, an inbox, and an optional wake schedule. Everything lives in one SQLite file, so assistants survive process restarts.

from orqest.assistants import AssistantRuntime, AssistantSpec, Contract, ContractRegistry

Two kinds

Kind Defined by Runs
code a handler object with async def handle(message, ctx), bound with register_code() one message at a time, FIFO per recipient (at most 20 per tick)
llm a serializable AssistantSpec passed to create() (needs an AgentFactory) a batch of due messages per model run (at most 20); one message at a time while a retry is due (an earlier failure whose backoff has elapsed)

Handlers live in memory, so re-register code assistants on every start; their state is persisted. Re-registering keeps the stored state_schema (a warning is logged if you pass a different one). An AssistantSpec carries the agent definition (agent.output_schema required, agent.output_type forbidden), a JSON-Schema state_schema, initial_state, schedule wakes and allowed_contracts. Specs are versioned: redesign() appends a new version, rollback() re-activates an earlier one, and history is append-only.

Messages and state

Handlers never write to the database directly. They use a WakeContext: ctx.update_state(patch) (RFC 7386 merge patch), ctx.send(...), ctx.reply(message, type, payload) and ctx.schedule_once(at, reason). These are buffered and applied by the runtime afterwards.

Messages are processed FIFO per recipient among the messages that are due. A message that failed waits out its retry backoff, and later messages to the same recipient are processed in the meantime, so a retried message can be overtaken.

rt.send() to a missing or retired address raises UnknownRecipient. Inside a code handler, a ctx.send() or ctx.reply() to an address that is not a registered, non-retired assistant fails the whole handled message at commit time: its state change and every other buffered effect are discarded, and the message is retried and eventually dead-lettered, not just the reply. In particular, an external sender such as "user" can send in via rt.send, but a handler that replies to it fails. Register a code assistant (for example "ui") as the address for user-originated messages.

The transactional-outbox guarantee

For each processed message, the new state, all outgoing messages, any scheduled wakes and the message's own completion are committed in one SQLite transaction. If the process crashes mid-handler, nothing is persisted. When the runtime next starts, the message goes back to pending and is processed again, unless that crashed attempt was its last one (max_attempts), in which case it is dead-lettered (last_error says it crashed the runtime) so a message that always kills the process cannot loop forever. No state change is ever visible without its messages, and no message is delivered without its state change. This is pinned by the property test tests/assistants/test_crash_safety.py.

The guarantee covers the database only. Delivery is at-least-once: a handler or an LLM tool may run more than once for the same message (after a crash, a timeout or a retried failure). Anything a handler does outside the WakeContext, such as calling an external API, can therefore happen more than once. Deduplicate such side effects by message.id.

Buffered handler data is validated before commit (JSON-serialisable, state is a dict of strict JSON with no NaN or infinity, state matches its schema; a schema violation names up to three failing locations as dot paths, e.g. at state.plan.weeks.1: 'label' is a required property). Initial state passed to register_code() or create() is held to the same strict-JSON rule (StateValidationError). Invalid data does not wedge the runtime: the message is retried with exponential backoff and, after max_attempts, dead-lettered, which emits an assistant.message.dead event on the EventBus. Each handler call or model run is bounded by run_timeout_s (default 300 s; None disables it). A run that exceeds it fails with TimeoutError and goes through the same retry and dead-letter path.

A CancelledError raised from a handler while its run is not actually being cancelled (for example a library awaiting a future that was cancelled) is treated as an ordinary failure. A real cancellation of tick() propagates. Any other BaseException from a handler (the crash path) propagates out of tick(), as do database errors from the runtime's own transactions (crash-early).

A row the runtime cannot decode (a corrupt assistant record or wake) does not stop every tick. A corrupt assistant record is skipped and its messages wait. A corrupt wake is deactivated. Each is logged once.

Scheduler

Wakes (Wake(kind="daily" | "weekly" | "once", ...)) turn into ordinary wake messages in the inbox.

  • A wake that was missed while the runtime was down fires once on the next tick, not once per missed occurrence.
  • A paused assistant's recurring wakes are skipped and advanced to their next occurrence. Its once wakes are still enqueued when due: the wake message waits in the inbox and is processed after resume().
  • A retired assistant's wakes are deactivated.
  • A once wake in an AssistantSpec.schedule whose time is already past is stored inactive and never fires. By contrast, rt.schedule_once() (or ctx.schedule_once()) with a past time fires on the next tick.

Contracts are the capability channel

A Contract declares one message type an address accepts, with a JSON-Schema payload and a description. Contracts are checked on rt.send() and on a handler's ctx.send() / ctx.reply(); violations raise ContractViolation. An address with no registered contracts accepts any message type and payload.

LLM assistants are structurally limited to:

  • the seven runtime tools update_state, reply_to_message, schedule_wake, publish_view, show_artifact, dismiss_artifact and describe_blocks, and
  • one generated tool per contract in the spec's allowed_contracts, named <address>__<message_type>, and
  • with a compartment grant and an AssistantRuntime(keeper=...), the tools that grant allows (compartment_read, compartment_prepare_release, compartment_write, compartment_delete). See Compartments.

AssistantSpec rejects any agent.tools, so an LLM assistant cannot gain other capabilities. reply_to_message is not a way around allowed_contracts: a reply to a sender that has registered contracts is allowed only if (sender, message_type) is in the spec's allowed_contracts. A sender with no registered contracts accepts anything, so any reply to it is allowed. If a tool targets an address that is missing or retired (or a reply target that is not a registered assistant), or breaks one of these rules, the tool returns an ERROR: ... string the model can correct; the commit-time check stays authoritative. schedule_wake accepts at most 10 calls per run; later calls return an error string.

Sends are asynchronous. A contract tool or reply_to_message only queues a message in the run's outbox; it is delivered when the run commits, after the wake ends. Any reply therefore arrives as a new message in a later wake, never as the tool's result. The tools say so: a contract tool returns queued: <id> (delivered after this wake ends; a reply arrives in a later wake — do not resend) and reply_to_message returns queued: <id> (delivered after this wake ends); the queued: <id> prefix is stable for consumers that parse it. The wake prompt's "How to act" section repeats it: never send the same request twice or call a tool again to check on it, batch independent calls in one response, then return the output to end the wake. As a backstop, an identical send repeated within one run (same address, message type and canonical JSON payload; for a reply, the same message id, type and payload) is not queued again: it returns already queued: <id> this wake — a reply arrives in a later wake; return your output now. A different payload is a different send. update_state returns only the top-level keys whose value changed (updated: plan, streak), or unchanged: ... when the patch changes nothing, rather than echoing the whole state into the model's context on every call. When a patch makes one of the assistant's views newly degrade a node to the "Unavailable" placeholder (a bound value the block's data rejects, such as a timeline event status outside planned | done | missed | cancelled, or a binding that no longer resolves), the patch still applies and the result says so: updated: booked (view problems: page/tl: invalid data: data.events.0.status: ...), at most three problems of at most 200 characters each. The check covers the views stored before the wake and those published during it (default fallback views are not checked), each resolved against the run's binding context with the old and then the new state; it skips views with no binding into state or a changed top-level key. It reports only problem messages that were not there before the patch, so it is approximate: a list-index shift can re-report an old break under its new index, and only the first data error per node is seen. The wrap-up suffix, when present, still comes last.

Wrapping up. "How to act" also says each wake has a limited number of model requests and to aim to finish in two or three. When the run has a request limit (the configured usage_limits.request_limit, or pydantic-ai's default 50 when usage_limits=None), every runtime and contract tool result produced once the run has made request_limit - 2 requests ends with [wake nearly out of model requests (<used> of <limit>): finish now — return your output]. It is a suffix only, so the queued:, already queued:, updated:, unchanged:, published:, scheduled: and ERROR: prefixes are unchanged. The tools read the count from pydantic-ai's RunContext, which is not a model-visible argument, so their JSON schemas are unchanged; build_runtime_tools takes the limit as a keyword-only request_limit (None, the default, never nudges).

The last allowed request. Once a run has made request_limit - 1 requests, the output tool is the only tool offered, so a model that keeps filling its wake has to end it. A run ending that way keeps its work and its messages are done. Calls in the same response as the output tool still run (end_strategy="exhaustive"); one with invalid arguments is dropped, because the run ends with that response. Nothing can be repaired on that last request: if its output fails validation, or the model answers with text or calls a hidden tool, the retry would be over the limit, so the run fails with UsageLimitExceeded, its work rolls back and it goes through the normal retry / dead-letter path.

Replies are optional. Each message in the wake prompt's ## New messages section carries id, sender, type, payload, reply_to and reply_with, which says what reply_to_message accepts for it: [] when the sender cannot receive messages; for a sender with registered contracts, the sorted message types the assistant's allowed_contracts let it send to that sender (possibly []); "any" for a reachable sender with no contracts (a user, another assistant). "How to act" tells the model to reply only when a message asks for an answer and reply_with allows it; results, confirmations and notices need no reply. A reply outside allowed_contracts returns ERROR: replying '<type>' to '<sender>' is not in your allowed_contracts — this message needs no reply; do not retry, and a target that cannot receive messages returns ERROR: '<address>' cannot receive messages — do not retry.

Assistants holding a read grant are checked on the way out. reply_to_message, contract tools, update_state, publish_view, show_artifact and a compartment_write label refuse anything holding a stored value with ERROR: this would reveal stored private information (<labels>); say what changed, not the private detail. ..., and nothing changes. A run whose output holds one is not remembered. It catches the literal value only; Compartments lists what it misses.

Each run recalls episodic memory (when a MemoryStore is configured): stored entries are prefixed with the batch's message types and recalled per message type. This is coarse by design.

Lifecycle

pause(), resume(), retire(), redesign() and rollback() take the same per-address lock as message runs, so they serialise with an in-flight run: a retire waits for the current run's commit or failure. Retiring fails the assistant's pending messages. Each of these, and create(), emits assistant.lifecycle {action, version} on the bus after it commits (action is create, pause, resume, retire, redesign or rollback).

Called from inside a handler, a lifecycle call on another address waits for that address's current run to finish. Two runs that target each other this way wait on each other until run_timeout_s fails one of them, so avoid that pattern. A lifecycle call on the handler's own address raises RuntimeError immediately, because it would wait for itself.

redesign(address, spec, migrate=...) accepts a sync or async migrate(old_state) -> new_state. The current state is read and migrated while the address lock is held, so no run of that assistant can commit in between. The new spec, state, version snapshot and wakes are then written in one transaction. If anything fails, nothing changes.

Screens

An assistant can own named views: declarative screens built from the UI blocks in Generative UI. Views are stored with the assistant (one per (address, name), replaced on republish) and committed in the same transaction as the run's state, so a view is never visible without the state that produced it.

  • Publishing. A code handler calls ctx.publish_view(ViewSpec(...)). An LLM assistant gets publish_view(name, root, version) and describe_blocks(types) (and show_artifact / dismiss_artifact, see Artifacts). describe_blocks returns {"blocks": {type: schema of its data}, "hints": ..., "view_node": {"schema", "note"}, "unknown": [...]}: the node schema plus a note on the binding grammar and the tree rules (root is a layout, ids unique). A block whose data annotation has no JSON schema gets null; names that are not registered go in unknown. These are the only view tools added; the structural limit on LLM capabilities is otherwise unchanged. Validation is structural only (unknown block type or key, a binding key that is not a data field, invalid data): a binding that does not resolve is not an error, it renders as an "Unavailable" placeholder and is returned as a warning (ctx.publish_view returns the warning lines; the tool answers e.g. published: today (warnings: binding 'state.x' did not resolve at node 'n')). ViewValidationError (exported from orqest.assistants, with a problems list) is raised to code handlers; the LLM tool returns an ERROR: ... string instead.
  • view_context. AssistantRuntime(view_context=builder) takes an async builder(record) -> dict that supplies the binding context. Its "state" key is replaced by the run's working state when a view is published, and by the committed state when views are emitted or rendered. The default context is {"state": record.state}. The builder is awaited once per run (its time counts against run_timeout_s), and again for every emission and render_view call, with a copy of the record. If it raises, returns a non-dict or exceeds view_context_timeout_s (default 5 s, capped by run_timeout_s; None leaves only that cap), a warning is logged (once per failure streak per address) and the default context is used, so a broken builder never dead-letters messages. One exception: when run_timeout_s is not larger than view_context_timeout_s, a hung builder is cut off by the run timeout itself and can still fail the run.
  • Live events. With a bus, after each commit the runtime resolves the address's stored views against the committed state and emits what changed (including ui.view.remove for a view that disappeared), on the EventBus, under the address lock: ui.view.init ({address, view, version, seq, tree}) and ui.view.delta ({address, view, version, seq, deltas}, using diff_view's top-level-field granularity). seq orders events per (address, view); renderers follow the Renderer contract. A view re-inits (full tree) when it has not been emitted yet, when its stored spec differs from the last emitted spec (even at the same version), or when diff_view returns None (name, version, block set or block type changed). redesign() and rollback() emit too, after their write. Nothing is emitted without a bus. Emission is best-effort: a failed emit is logged, never fails the commit, and the next emission re-inits that view. The last-emitted cache is in memory, so the first emission after a restart is always an init.
  • Snapshots. await rt.render_view(address, name) returns a ViewSnapshot (resolved, seq; exported from orqest.assistants) for a renderer that is (re)loading. When the view has been emitted, it is the last-emitted resolution and its seq, so a renderer that subscribed first can apply exactly the events after it. Otherwise it resolves the stored view fresh (committed state and view_context; fallback_view(name) when no such view is stored) with seq 0, and the next event for that view is an init. It does not change the emission cache, and raises UnknownRecipient for a missing or retired assistant.
  • Listing and refreshing. await rt.list_views(address) returns the stored view names plus, for an llm assistant, its default_views names, sorted. await rt.refresh_views(address) re-resolves the views and emits what changed, as after a commit: use it when view_context data changed outside any run. It takes the address lock (waiting for an in-flight run), does nothing more without a bus, raises UnknownRecipient for a missing or retired assistant, and raises RuntimeError when called from inside that assistant's own run.
  • default_views. AssistantRuntime(default_views=("home", ...)) names views every llm assistant shows: each one it has not published is a fallback_view of its state. They are emitted, rendered and listed like stored views, and a stored view of the same name wins. Code assistants get no fallback screens (a fallback would show a service's raw state, tokens included); the views they publish themselves still work. Every name is validated when the runtime is constructed: an invalid view name raises ValueError. The default is ().
  • Version snapshots and rollback. redesign() and rollback() save the assistant's current views under the version being replaced. rollback(address, N) restores the views that were in place when version N was last replaced: views absent from that snapshot are removed, and for each one the runtime emits ui.view.remove ({address, view, seq}) and drops its cache entry. If N has no saved views (never replaced, or it had none), the current views are left as they are.
  • Lifecycle guard during emission. Emission runs as the address's own run, so a view_context builder that calls pause, retire, redesign, rollback or refresh_views on that address gets RuntimeError instead of waiting on the lock emission holds. Tasks the builder spawns inherit the guard (it is a context variable).
  • Media. MediaComponent URLs must be https:// (the PWA is served over HTTPS, so http:// media would be blocked as mixed content); anything else fails validation.
  • Escape hatch. A view node of type sandboxed_html emits a ui.view.escape_hatch event ({address, view, node_id}) once per (address, view, node) per process, so a host can notice that an assistant stepped outside the typed blocks.

Running it

Call tick() yourself (tests, scripted loops), or let run_forever(poll_interval_s=0.5) do it.

tick() blocks: recipients run concurrently, but it returns the number of completed messages only when its slowest recipient has finished, so one slow assistant delays the next tick for everyone by up to run_timeout_s per message (plus up to another run_timeout_s for view emission when a bus is set; a code assistant runs up to 20 messages per tick).

run_forever does not have that coupling. It supervises one worker task per address: each iteration reaps finished workers, fires due wakes, lists the recipients with due messages and starts a worker (one run, as in a tick) for every one that has none, never two for the same address. A slow assistant therefore never delays the others. It sleeps poll_interval_s only when nothing started or finished (waking early when a worker finishes), and also when the workers that finished completed no message between them (failed runs, or a recipient that was listed but had nothing to claim), so a failing recipient is retried once per poll rather than in a hot loop. A backlog drains without delay.

Cancelling the run_forever task cancels and awaits every worker, then raises CancelledError (standard asyncio). Any other exception, from a worker or from the runtime's own transactions, is fatal: the sibling workers are cancelled and awaited, then it propagates and run_forever ends. A worker that ends cancelled while run_forever itself is not being cancelled (for example a CancelledError escaping a bus subscriber) is fatal too: run_forever raises RuntimeError("worker '<address>' cancelled unexpectedly") instead of looking like a clean stop. Only the first failure propagates; every other worker that crashed, or raised while being cancelled, is logged at ERROR.

run_forever starts by returning every processing message to pending (in one transaction), so restarting it in the same open runtime, after a cancel or a fatal error, resumes the messages the previous loop left in flight. That is also why two loops must never share a database:

  • One runtime per database file. Do not run several processes over the same file (no multi-worker uvicorn --workers N or gunicorn -w N; use one worker), and do not open two runtimes on one file in a process.
  • Never run tick() and run_forever concurrently, nor two run_forever loops: the second one's start-up recovery would hand the first one's in-flight messages out again.

Bus subscribers run inline, inside the run that emits the event (runtime events such as assistant.message.dead and assistant.run.completed, and live view events). Keep them quick: a slow subscriber holds that assistant's address lock and delays its next run. A subscriber must never call a lifecycle method (pause, resume, retire, redesign, rollback, refresh_views) on the event's own address: inside the run it would wait for itself, so the call raises RuntimeError (also from tasks the subscriber spawns, which inherit the run's context). Put such work on a queue that code outside the runtime's runs consumes.

Mapping the spec's "watchdogs" to this runtime: a stuck run is bounded by run_timeout_s, a looping run by usage_limits (below), and a model outage is handled by the host passing a FallbackModel as model or from model_for.

Shut down in this order: cancel the run_forever task and await it, then leave the runtime's async with block. Leaving the block first closes the database under the running loop, which then fails with RuntimeError.

async with AssistantRuntime("assistants.db") as rt:
    ...  # register handlers
    runner = asyncio.create_task(rt.run_forever())
    try:
        ...  # serve until shutdown
    finally:
        runner.cancel()
        with contextlib.suppress(asyncio.CancelledError):
            await runner

Embedded in a web server, a fatal error ends run_forever while the server keeps serving, so watch the task: a done-callback that logs the error and exits the process lets systemd (or whatever supervises the process) restart it, and the next start recovers every in-flight message. A FastAPI-style lifespan:

import asyncio
import contextlib
import os

from loguru import logger


def exit_on_fatal(task: asyncio.Task) -> None:
    if task.cancelled():
        return  # normal shutdown
    if (exc := task.exception()) is not None:
        logger.opt(exception=exc).critical("assistants loop died: {e!r}", e=exc)
        os._exit(1)  # non-zero: the process supervisor restarts us


@contextlib.asynccontextmanager
async def lifespan(app):
    async with AssistantRuntime("assistants.db", bus=bus) as rt:
        ...  # register handlers
        app.state.assistants = rt
        runner = asyncio.create_task(rt.run_forever())
        runner.add_done_callback(exit_on_fatal)
        try:
            yield
        finally:
            runner.cancel()
            with contextlib.suppress(asyncio.CancelledError):
                await runner

The database records its schema version in PRAGMA user_version (currently 1). There is no migration code yet.

Models, limits and token budgets

  • model and model_for. model is the runtime-wide default. model_for(record, batch) -> Model | None is called once per llm run and may route per assistant or per message type; returning None falls back to model, then to the model in the spec.
  • usage_limits. Every llm run is bounded by a pydantic-ai UsageLimits, default UsageLimits(request_limit=20, tool_calls_limit=40). The last allowed request offers only the output tool, so a run reaching request_limit normally ends there with its work kept (see The last allowed request). A run that fails on that request, or that hits tool_calls_limit, fails and goes through the normal retry / dead-letter path. usage_limits=None does not remove the bound: pydantic-ai then applies its own default UsageLimits() (request_limit=50). This is the guard against looping runs.
  • assistant.run.completed. After a successful llm run commits, with a bus the runtime emits it with data {"address", "messages", "model", "input_tokens", "output_tokens", "requests", "tool_calls"}. model is the model that actually answered: the model name on the run's final response (so a FallbackModel reports the fallback that replied, and a provider its dated snapshot), else the name of the model the agent was given.
  • What is charged. Input plus output tokens of every llm run are added to the assistant's day in the usage table: on success, on failure and on cancellation alike. A run is charged when it ends, so a run spanning midnight (in budget_tz) counts towards the day it ends on. Charging is best-effort: a failure to record is logged and never fails the commit, and a run in flight when the process crashes is never charged.
  • Budgets. AssistantSpec.daily_token_budget (or AssistantRuntime(daily_token_budget=...) as the default for specs that set none; None means no limit) caps tokens per day. An exhausted budget holds messages, it does not pause the assistant: its due messages stay unclaimed and the assistant runs again by itself once the day rolls over. Nothing needs resuming. The day is the ISO date in budget_tz (default "UTC"; an unknown zone raises ValueError). A budget also binds inside a run: the run's UsageLimits get total_tokens_limit set to what is left of today's budget (the configured usage_limits keep their request / tool-call limits, and the smaller token limit wins). pydantic-ai checks the limit after each model response, so the response that crosses it still counts and the day can overshoot by that response. A run stopped by the budget limit is charged at once, and its messages go back to pending without counting the attempt (Inbox.release, which sets last_error to "held: daily token budget exhausted"): the budget blocker then holds them until the next day, so they are never dead-lettered for running out of budget. Nothing of the stopped run is kept, so if even its final response crossed the remaining budget, that work is lost and the whole batch runs again the next day. If the charge cannot be stored, the tokens still count in memory (the budget holds in this process) but the attempt counts too, since a restart would forget them. One exception: a run that already had the whole day's budget and still crossed it would fail the same way every day, so it counts the attempt like any other failure and eventually dead-letters. A UsageLimitExceeded from the configured limits is an ordinary failure too.
  • assistant.budget.exhausted. {"address", "day", "used", "budget"}, emitted the first time the runtime finds an exhausted assistant with due messages on a given day. The "already announced" memory is in process, so after a restart the same (address, day) can be announced again: consumers dedupe on (address, day). The blocked recipient is also logged once per exhausted day.
  • Daily spend cap. AssistantRuntime(spend_cap=SpendCap(daily, price, priority=frozenset(), reserve=0.0)) caps what all assistants spend together per day (the day is budget_tz's). Orqest knows no currency: price(model_name, usage) turns a run's RunUsage into an amount in daily's unit; model_name is the answering model's, falling back for a failed or cancelled run to the model's own name or the spec's model string (None if unknown). SpendCap raises ValueError unless daily > 0 and 0 <= reserve < daily. An address is blocked once spent >= daily, or, outside priority, once spent >= daily - reserve: the last reserve of the day belongs to the priority addresses. A blocked llm assistant's messages stay pending with no attempt counted, exactly like a token-budget hold, and run on the next day; code assistants always run. When several reasons block an assistant the first of contracts, token budget, spend is reported (SPEND_EXHAUSTED = "daily spend cap reached"). Runs are charged when they end (failed and cancelled ones too), so without run_reserve every run that starts while money remains can overshoot: up to one run per assistant running at once. With SpendCap(run_reserve=...) the runtime holds that amount (memory only; a restart starts with none) for each llm run from its claim until it is charged, and blocks an address when spent + held + run_reserve would pass its limit, so a run overshoots only by its actual cost above the reservation; a run that cannot take a hold leaves its messages pending without an attempt. assistant.spend.exhausted and the once-a-day warning fire when spent + run_reserve passes the limit, ignoring what is held for runs in flight, so a day that cannot fit another run is announced while a short wait for an in-flight run is not. ValueError if run_reserve is negative or above daily. If price raises (or returns a negative or non-finite number) the error is logged at ERROR and the rest of the day is charged, so a broken price cannot lift the cap. A failed write is logged and still counts in memory. For model calls outside an llm wake (an application's own direct calls) use await runtime.charge(address, usage, model_name), which adds the tokens to the token budget and the amount to the spend and returns the amount; it is safe to call from a code handler. A failed write raises, but the amount still counts in this process (a restart loses it), so a handler that lets it propagate is retried and charges again. runtime.spend_blocker(address) is the synchronous check before such a call (it follows the same rule as the event and ignores amounts held for runs in flight, so a passing run cannot make "the budget is used" false; only the start of an llm run counts holds), and await runtime.spend_today() returns a SpendSummary(day, spent, daily, by_address) (None without a cap). assistant.spend.exhausted carries {"day", "spent", "daily", "scope"}, with scope "background" while only non-priority addresses are blocked and "all" at the full cap, once per scope per day in a process.
  • Wake context. AssistantRuntime(wake_context=async (record, batch) -> dict | None) is called once per llm wake, inside it and before the prompt is built, so what it returns is current and is never stored in a message. A dict becomes a ## Context section (its JSON) directly after ## New messages; None leaves the section out, and a runtime without the callback renders the same prompt bytes as before. An exception, a result that is not a JSON-serializable dict, or a call taking longer than 5 s (runtime.WAKE_CONTEXT_TIMEOUT_S) logs a warning with the exception type (not its text) and leaves the section out; the wake still runs. The context is data the application supplied: the model is told nothing beyond the heading, and it is not leak-checked, so the application must not put in it what an assistant without the matching grant may not see.

Example: two code assistants exchanging a message

import asyncio
import tempfile
from pathlib import Path

from orqest.assistants import AssistantRuntime


class Echo:
    """Counts messages and replies to the sender."""

    async def handle(self, message, ctx):
        ctx.update_state({"count": ctx.state["count"] + 1})
        ctx.reply(message, "echoed", {"n": ctx.state["count"]})


class Sink:
    """Records every message it receives."""

    async def handle(self, message, ctx):
        ctx.update_state({"got": ctx.state["got"] + [{"type": message.type, **message.payload}]})


async def main() -> None:
    with tempfile.TemporaryDirectory() as tmp:
        async with AssistantRuntime(Path(tmp) / "assistants.db") as rt:
            await rt.register_code("echo", Echo(), initial_state={"count": 0})
            await rt.register_code("sink", Sink(), initial_state={"got": []})

            await rt.send("sink", "echo", "ping", {})
            await rt.tick()  # echo handles "ping" and replies to sink
            await rt.tick()  # sink handles the reply
            print((await rt.get("sink")).state)
            # {'got': [{'type': 'echoed', 'n': 1}]}


asyncio.run(main())