Skip to content

Generative UI API

ui

Generative UI primitives — agents that design their own surface.

The agent emits a typed :class:UIComponentSpec; the frontend resolves a renderer by component_type and hot-loads it. Subsequent state updates flow as :class:UIDeltaEvent records on the same SSE stream.

See :doc:/concepts/generative_ui for the full picture; this module hosts:

  • :class:UIComponentSpec[T] + :class:UIDeltaEvent + :data:UIDeltaOp — the protocol primitives.
  • :class:ComponentRegistry — per-Workbench schema registry.
  • :class:UIEmitter — convenience facade for emitting init/delta/remove events on an :class:EventBus.

  • First-party components organised in three layers:

  • Compositional primitives: :class:PlanComponent, :class:ChartComponent, :class:TableComponent, :class:FormComponent, :class:TakeoverDialogComponent, :class:LayoutComponent, :class:TextComponent, :class:MarkdownComponent, :class:ImageComponent, :class:BadgeComponent, :class:ButtonComponent, :class:InputComponent.

  • Declarative grammars: :class:VegaChartComponent, :class:MermaidComponent, :class:LatexComponent, :class:JsonViewerComponent.
  • Sandboxed escape hatch: :class:SandboxedHTMLComponent.

UIDeltaOp module-attribute 

UIDeltaOp = Literal['replace', 'merge', 'append', 'remove']

Op set for partial component updates.

  • replace — set value at path to value (RFC 6902 replace).
  • merge — shallow-merge value (a dict) into the dict at path; if value-at-path is not a dict, behaves like replace.
  • append — append value to the list at path. Required for streaming-append components (chart series, table rows).
  • remove — delete the field/element at path. value ignored.

BadgeComponentData

icon class-attribute instance-attribute 
icon: str | None = None

lucide-react icon name. The frontend resolves the symbol; if the icon is unknown it falls back to label-only.

ButtonComponentData

event_name class-attribute instance-attribute 
event_name: str = 'ui.button.clicked'

The event_type the frontend POSTs back when the button is clicked. The agent subscribes to this via the SSE bus + event-loop integration.

ChartComponentData

config class-attribute instance-attribute 
config: dict[str, Any] = Field(default_factory=dict)

Free-form renderer-specific knobs (axis formatters, colour scheme, etc.). Not load-bearing for the protocol.

ChartSeries

One labelled data series. Points are dicts so the renderer can pick out the relevant axes (e.g. {"x": ..., "y": ...}).

FormComponentData

submit_event class-attribute instance-attribute 
submit_event: str = 'form.submitted'

The event_type the frontend should emit (back to the agent) on submission. Decoupled from the component_type so multiple forms can share a single rendering path.

FormField

options class-attribute instance-attribute 
options: list[str] = Field(default_factory=list)

For select / multiselect — the available choices.

InputComponentData

rows class-attribute instance-attribute 
rows: int | None = None

Textarea row count (ignored for other kinds).

accept class-attribute instance-attribute 
accept: str | None = None

File mime/extension filter (e.g. "image/*" or ".csv,.tsv").

JsonViewerComponentData

data class-attribute instance-attribute 
data: Any = None

Any JSON-serialisable structure (object, array, scalar).

expanded_paths class-attribute instance-attribute 
expanded_paths: list[str] = Field(default_factory=list)

Paths to expand by default (e.g. ["", "items.0"]).

LatexComponentData

content instance-attribute 
content: str

LaTeX source rendered via KaTeX. Use display=True for block math, False for inline.

LayoutComponentData

gap class-attribute instance-attribute 
gap: int = 8

Tailwind spacing unit. 8 ≈ 0.5rem (matches existing Polymath UI).

grid_columns class-attribute instance-attribute 
grid_columns: int | None = None

Number of columns for direction="grid" (ignored otherwise).

MarkdownComponentData

content instance-attribute 
content: str

GFM markdown — tables, code, links, images. Frontend uses react-markdown + remark-gfm + the existing CodeBlock renderer.

MermaidComponentData

diagram instance-attribute 
diagram: str

Mermaid source. flowchart / sequence / classDiagram / erDiagram / gantt / mindmap / etc.

PlanComponentData

Data payload — list of :class:PlanTask records.

SandboxedHTMLComponentData

html instance-attribute 
html: str

Raw HTML / SVG / restricted JS. Rendered in an iframe with sandbox="allow-scripts" and a strict CSP. No parent-document access, no network unless allowed via the iframe csp.

height_px class-attribute instance-attribute 
height_px: int = 400

Fixed height in CSS pixels — iframes don't auto-size.

csp_extra class-attribute instance-attribute 
csp_extra: str = ''

Additional CSP directives merged onto the default. Empty by default.

TableColumn

key instance-attribute 
key: str

Row-dict key the renderer reads to populate this column.

TakeoverDialogData

choices class-attribute instance-attribute 
choices: list[str] = Field(default_factory=list)

For choice kind — the offered options.

VegaChartComponentData

spec class-attribute instance-attribute 
spec: dict[str, Any] = Field(default_factory=dict)

Full Vega-Lite spec (https://vega.github.io/vega-lite/). The frontend imports vega-embed and renders. The agent emits the JSON directly — Vega-Lite syntax is well-documented and the LLM is typically familiar with it.

UIEmitter 

UIEmitter(
    bus: EventBus | None = None, *, agent_name: str = "ui"
)

Facade for publishing :class:UIComponentSpec and :class:UIDeltaEvent events on a :class:EventBus.

Source code in orqest/ui/emitter.py 
33
34
35
36
37
38
39
40
def __init__(
    self,
    bus: EventBus | None = None,
    *,
    agent_name: str = "ui",
) -> None:
    self._bus = bus
    self._agent_name = agent_name
init async 
init(
    component: UIComponentSpec[Any],
    *,
    agent_name: str | None = None,
) -> AgentEvent | None

Emit ui.<component_type>.init for a fresh component.

Returns the emitted :class:AgentEvent for the caller to log / record, or None when no bus is configured. Bus failures log at DEBUG and return None.

Source code in orqest/ui/emitter.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
async def init(
    self,
    component: UIComponentSpec[Any],
    *,
    agent_name: str | None = None,
) -> AgentEvent | None:
    """Emit ``ui.<component_type>.init`` for a fresh component.

    Returns the emitted :class:`AgentEvent` for the caller to log /
    record, or ``None`` when no bus is configured. Bus failures
    log at DEBUG and return ``None``.
    """
    event = AgentEvent(
        event_type=ui_init_event_type(component.component_type),
        agent_name=agent_name or self._agent_name,
        data=component.to_event_data(),
    )
    if self._bus is None:
        return event
    try:
        await self._bus.emit(event)
    except Exception as exc:
        logger.debug("UIEmitter.init failed: {e}", e=exc)
        return None
    return event
delta async 
delta(
    *,
    component_id: str,
    component_type: str,
    op: UIDeltaOp,
    path: str = "",
    value: Any = None,
    agent_name: str | None = None,
) -> AgentEvent | None

Emit ui.<component_type>.delta with a partial update.

Source code in orqest/ui/emitter.py 
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
async def delta(
    self,
    *,
    component_id: str,
    component_type: str,
    op: UIDeltaOp,
    path: str = "",
    value: Any = None,
    agent_name: str | None = None,
) -> AgentEvent | None:
    """Emit ``ui.<component_type>.delta`` with a partial update."""
    delta = UIDeltaEvent(
        component_id=component_id,
        component_type=component_type,
        op=op,
        path=path,
        value=value,
    )
    event = AgentEvent(
        event_type=ui_delta_event_type(component_type),
        agent_name=agent_name or self._agent_name,
        data=delta.to_event_data(),
    )
    if self._bus is None:
        return event
    try:
        await self._bus.emit(event)
    except Exception as exc:
        logger.debug("UIEmitter.delta failed: {e}", e=exc)
        return None
    return event
remove async 
remove(
    *,
    component_id: str,
    component_type: str,
    agent_name: str | None = None,
) -> AgentEvent | None

Emit ui.<component_type>.remove so the frontend unmounts the component.

Source code in orqest/ui/emitter.py 
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
async def remove(
    self,
    *,
    component_id: str,
    component_type: str,
    agent_name: str | None = None,
) -> AgentEvent | None:
    """Emit ``ui.<component_type>.remove`` so the frontend
    unmounts the component."""
    event = AgentEvent(
        event_type=ui_remove_event_type(component_type),
        agent_name=agent_name or self._agent_name,
        data={"component_id": component_id},
    )
    if self._bus is None:
        return event
    try:
        await self._bus.emit(event)
    except Exception as exc:
        logger.debug("UIEmitter.remove failed: {e}", e=exc)
        return None
    return event

ComponentRegistry 

ComponentRegistry()

Maps component_type → :class:UIComponentSpec subclass.

Source code in orqest/ui/registry.py 
29
30
def __init__(self) -> None:
    self._specs: dict[str, type[UIComponentSpec[Any]]] = {}
register 
register(
    spec_class: type[UIComponentSpec[Any]],
    *,
    overwrite: bool = False,
) -> None

Register a concrete :class:UIComponentSpec subclass.

Reads the component_type Literal default off spec_class; the discriminator must be set on the class (not the instance). Re-registering the same component_type is a no-op unless overwrite=True (the prior class is logged at WARN).

Source code in orqest/ui/registry.py 
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
def register(
    self,
    spec_class: type[UIComponentSpec[Any]],
    *,
    overwrite: bool = False,
) -> None:
    """Register a concrete :class:`UIComponentSpec` subclass.

    Reads the ``component_type`` Literal default off ``spec_class``;
    the discriminator must be set on the class (not the instance).
    Re-registering the same ``component_type`` is a no-op unless
    ``overwrite=True`` (the prior class is logged at WARN).
    """
    component_type = self._extract_type(spec_class)
    if component_type in self._specs and not overwrite:
        logger.warning(
            "Component {ct} already registered; pass overwrite=True to replace",
            ct=component_type,
        )
        return
    self._specs[component_type] = spec_class
get 
get(
    component_type: str,
) -> type[UIComponentSpec[Any]] | None

Return the registered spec class for component_type, or None.

Source code in orqest/ui/registry.py 
54
55
56
def get(self, component_type: str) -> type[UIComponentSpec[Any]] | None:
    """Return the registered spec class for ``component_type``, or ``None``."""
    return self._specs.get(component_type)
list_types 
list_types() -> list[str]

All registered component types, sorted.

Source code in orqest/ui/registry.py 
58
59
60
def list_types(self) -> list[str]:
    """All registered component types, sorted."""
    return sorted(self._specs.keys())
validate_payload 
validate_payload(
    component_type: str, payload: dict[str, Any]
) -> UIComponentSpec[Any] | None

Hydrate a raw dict against the registered spec.

Returns None on lookup miss or validation failure (logged at WARN). Best-effort — UI validation should never break agent execution.

Source code in orqest/ui/registry.py 
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def validate_payload(
    self, component_type: str, payload: dict[str, Any]
) -> UIComponentSpec[Any] | None:
    """Hydrate a raw dict against the registered spec.

    Returns ``None`` on lookup miss or validation failure (logged
    at WARN). Best-effort — UI validation should never break agent
    execution.
    """
    cls = self._specs.get(component_type)
    if cls is None:
        return None
    try:
        return cls.model_validate(payload)
    except Exception as exc:
        logger.warning(
            "Component validation failed for {ct}: {e}", ct=component_type, e=exc,
        )
        return None

UIComponentSpec

Generic init payload for a frontend-renderable component.

Subclasses MUST override component_type with a unique :class:Literal value so the frontend resolver can pick a renderer. The data field carries the typed payload the renderer reads.

The schema is intentionally minimal: every concrete component inherits the same envelope (component_type, component_id, data, metadata, created_at) so the SSE protocol and the frontend resolver can be component-agnostic.

to_event_data 
to_event_data() -> dict[str, Any]

Serialize to the dict that lands inside :class:AgentEvent.data.

Source code in orqest/ui/spec.py 
53
54
55
def to_event_data(self) -> dict[str, Any]:
    """Serialize to the dict that lands inside :class:`AgentEvent.data`."""
    return self.model_dump(mode="json")

UIDeltaEvent

Targeted partial update to a previously-emitted :class:UIComponentSpec.

path is a dot-path into data (root = empty string). For list operations, integer indices may appear in the path (e.g. "tasks.0.status"). Frontend implementations apply the op against the previously-rendered component identified by component_id; an unknown id should be treated as a no-op (the consumer can re-fetch via the snapshot endpoint to recover).

default_registry 

default_registry() -> ComponentRegistry

A :class:ComponentRegistry pre-loaded with first-party components.

Three layers of generative-UI primitives ship in core:

  1. Compositional — :class:PlanComponent, :class:ChartComponent, :class:TableComponent, :class:FormComponent, :class:TakeoverDialogComponent, :class:LayoutComponent, :class:TextComponent, :class:MarkdownComponent, :class:ImageComponent, :class:BadgeComponent, :class:ButtonComponent, :class:InputComponent. The agent composes these to build arbitrary UI.
  2. Declarative grammars — :class:VegaChartComponent, :class:MermaidComponent, :class:LatexComponent, :class:JsonViewerComponent. Thin wrappers around external grammars; the agent emits the spec.
  3. Sandboxed escape hatch — :class:SandboxedHTMLComponent. The agent writes raw HTML/SVG; the frontend confines it in an iframe. The component registers in core unconditionally; consumers gate rendering / emission via their own config flag.
Source code in orqest/ui/registry.py 
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
def default_registry() -> ComponentRegistry:
    """A :class:`ComponentRegistry` pre-loaded with first-party components.

    Three layers of generative-UI primitives ship in core:

    1. **Compositional** — :class:`PlanComponent`, :class:`ChartComponent`,
       :class:`TableComponent`, :class:`FormComponent`,
       :class:`TakeoverDialogComponent`, :class:`LayoutComponent`,
       :class:`TextComponent`, :class:`MarkdownComponent`,
       :class:`ImageComponent`, :class:`BadgeComponent`,
       :class:`ButtonComponent`, :class:`InputComponent`. The agent composes
       these to build arbitrary UI.
    2. **Declarative grammars** — :class:`VegaChartComponent`,
       :class:`MermaidComponent`, :class:`LatexComponent`,
       :class:`JsonViewerComponent`. Thin wrappers around external
       grammars; the agent emits the spec.
    3. **Sandboxed escape hatch** — :class:`SandboxedHTMLComponent`. The
       agent writes raw HTML/SVG; the frontend confines it in an iframe.
       The component registers in core unconditionally; consumers gate
       *rendering* / *emission* via their own config flag.
    """
    from orqest.ui.components import (
        BadgeComponent,
        ButtonComponent,
        ChartComponent,
        FormComponent,
        ImageComponent,
        InputComponent,
        JsonViewerComponent,
        LatexComponent,
        LayoutComponent,
        MarkdownComponent,
        MermaidComponent,
        PlanComponent,
        SandboxedHTMLComponent,
        TableComponent,
        TakeoverDialogComponent,
        TextComponent,
        VegaChartComponent,
    )

    reg = ComponentRegistry()
    # Existing first-party components.
    reg.register(PlanComponent)
    reg.register(ChartComponent)
    reg.register(TableComponent)
    reg.register(FormComponent)
    reg.register(TakeoverDialogComponent)
    # Layer 1 — compositional primitives.
    reg.register(LayoutComponent)
    reg.register(TextComponent)
    reg.register(MarkdownComponent)
    reg.register(ImageComponent)
    reg.register(BadgeComponent)
    reg.register(ButtonComponent)
    reg.register(InputComponent)
    # Layer 2 — declarative grammars.
    reg.register(VegaChartComponent)
    reg.register(MermaidComponent)
    reg.register(LatexComponent)
    reg.register(JsonViewerComponent)
    # Layer 3 — sandboxed escape hatch (frontend gates rendering).
    reg.register(SandboxedHTMLComponent)
    return reg