Changelog¶
All notable changes to Orqest are documented here. The format follows Keep a Changelog and the project follows Semantic Versioning.
[Unreleased]¶
Security¶
-
Sandbox Phase 1 — defense-in-depth hardening (#8). Five focused commits closing documented and undocumented holes in
orqest.sandbox. None of the changes break the public API or constructor signatures; behavior changes are strictly "things that were quietly unsafe now refuse to run." -
Static validator widened.
_static.pyblocks reflection helpers that previously let LLM-emitted code reach dunder attributes by string lookup:getattr,setattr,delattr,hasattr,type,dir,super,__build_class__. Forbidden-attributes set adds__dict__,__init_subclass__,__init__,__new__. A newast.Subscriptcheck catchesobj["__class__"]-style string-keyed reach-through. -
Subprocess wrappers restrict
__builtins__. Tier-1 (SubprocessSandbox) and Tier-2 (in-containerExecutor) previously ran user code with the full Python builtin set —getattr/type/openreachable at runtime even when the static validator had a gap. Neworqest/sandbox/_safe_builtins.pycarries the curated set + a factory; every tier now builds its__builtins__from the same allowlist. The wrappers load_safe_builtins.py+_static.pyby file path (importlib.util.spec_from_file_location) to bypass the heavyorqest/__init__.pyimport chain that would blow past the 128 MBRLIMIT_AScap. -
Path-traversal hardening on identifiers. New
orqest/sandbox/_identifiers.pyexposes the grammar^[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}$and a fail-fast check. Wired in at three layers:DockerSandbox.__init__validatesuser_id/session_idbefore any Docker call;Executor.executevalidatesagent_idat the in-container boundary. Closesagent_id="../escape"paths that would have resolved out of the per-agent workspace. -
JWT scope separation. New
scopeclaim on the bearer JWT —"agent"(LLM-facing connection — whatDockerSandbox._mint_jwtstamps by default) or"operator"(host-orchestrator only).SessionAuthMiddlewareenforces a per-tool allowlist:promote_toolandforget_toolrejectagent-scope tokens. Tokens minted before this change (noscopeclaim) default to agent-scope — least privilege. NewDEFAULT_OPERATOR_TOOLS/SCOPE_AGENT/SCOPE_OPERATORconstants indocker_runtime/auth.py; new public helperDockerSandbox.mint_operator_token()for host code that needs to call persistence APIs. -
Origin enforcement on by default.
SessionAuthMiddleware.from_envdefaultsORQEST_ALLOWED_ORIGINStohttp://127.0.0.1,http://localhostwhen unset (was: empty = skip check). DNS-rebinding defense matches the MCP spec out of the box. Explicit empty value ("") is the documented escape hatch. NewDEFAULT_ALLOWED_ORIGINSconstant. -
Concurrency hardening.
asyncio.Lockaround the promotion threshold check inbuild_server(prevents duplicatepromote_toolfires on concurrent invocations of the same(name, hash)). Per-agentasyncio.LockaroundExecutor.ensure_venv+install_deps(preventsuv venvfrom racing when multiple concurrentexecute_pythoncalls target the sameagent_id). The execution subprocess itself stays outside the lock so cross-agent concurrency is preserved. -
Test resilience for optional
optimizationdeps (#9).tests/optimization/test_adapter.pyandtests/optimization/test_runner.pypreviously failed with confusing errors when theoptimizationdep group was absent from the venv (OrqestEvalBatch.__init__() got an unexpected keyword argument 'outputs'from the stubbedEvaluationBatchbase;ModuleNotFoundError: litellmfrom tests patchinglitellm.completion). Addedpytest.importorskip("gepa")at module level ontest_adapter.pyand per-testpytest.importorskip("litellm")on the twoTestMakeReflectionLMtests that touch it. Test outcomes now reflect test code, not optional-dep installation state — 1096 pass / 4 skip withoutgepa[full]; 1107 pass / 0 fail with it.
Removed¶
-
examples/directory — deleted entirely. The folder was a half-finished third venue for API exercises: theexecuted_*parallel notebooks in three subdirs had drifted from their sources (06 had 21 cells in the source vs 28 in the executed copy), example 05 was an out-of-format Python script while the rest were notebooks, and coverage stopped at the basic-orchestration primitives — none of the flagship features (AgentSpec,Sandbox,MetaOrchestrator,OptimizationRunner,RuntimeTopologyDesigner) had an example. The notebooks tour now carries the composed-story surface and thedocs/concepts/*.mdpages carry the primitive-by-primitive reference; cleaning up examples removes the third drift surface. Six inbound references updated:README.md,CLAUDE.md(×2),notebooks/README.md,docs/notebooks.md,orqest/skills/orqest/SKILL.md. -
SKILLS.mdat the repository root — superseded bydocs/concepts/skills.mdand the bundled skill atorqest/skills/orqest/SKILL.md. The single inbound reference inREADME.mdnow points at the docs page and the bundled skill. demo/directory (Polymath reference consumer) — not part of the published package and not relevant for PyPI consumers. The five remaining inbound references inREADME.md,CLAUDE.md,docs/concepts/autonomy.md, and.claude/skills/principles/SKILL.mdwere updated or removed.orqest/benchmarks/— internal BIRD and Spider benchmark code that shipped in the wheel but had no callers outside itself. Moved tobenchmarks/bird/andbenchmarks/spider/(withgit mvto preserve history), matching the project convention established bybenchmarks/coding/. Internalfrom orqest.benchmarks.*imports were converted to relative imports..claude/skills/gstackuntracked — was a symlink to an absolute path on the maintainer's machine; broken for any other clone. Added to.gitignoreso the local convenience symlink can stay.
Docs¶
- Documentation site polish landed in #6: rewrote
docs/index.mdfor v0.8.0; replaceddocs/changelog.mdwith anmkdocs-include-markdown-plugininclude directive that pulls the rootCHANGELOG.md; addeddocs/concepts/reasoning.mdanddocs/concepts/skills.md; addednavigation.tabs, social icons,git-revision-date-localized, and several markdown extensions tomkdocs.yml.
[0.8.0] - 2026-05-23¶
Added¶
-
Bundled
orqestskill for Claude Code / Cursor / Aider / other agentic IDEs — ships atorqest/skills/orqest/as package data.SKILL.md(orientation + decision tree + 5 wire-up patterns + pitfalls) plus 9 compressed judgment-layer references covering every battery:references/{orchestration,memory,autonomy,healing,metacognition,mcp,optimization,generative-ui,sandbox}.md. References point back atdocs/concepts/*.mdfor full depth — single source of truth for API surface stays in concept docs. Install into a project's skills directory withpython -m orqest.skills install [target=.claude/skills]. Drift-checked bytests/skills/test_skill_drift.py— everyfrom orqest…import in the skill resolves against the real package on each run. -
openai-responses:provider prefix inorqest.utils.resolve_model— routes topydantic_ai.models.openai.OpenAIResponsesModelsoLLM_MODEL=openai-responses:gpt-5.4(etc.) goes through the OpenAI Responses API instead of/v1/chat/completions. Required for gpt-5.x with function tools + reasoning_effort (chat/completions rejects that combo with a 400). Uses the sameOPENAI_API_KEY. Discovered while building Polymath's Koopman research session — gpt-5.4 + tools + reasoning silently desynchronised the message stream on chat/completions; switching to the Responses path resolved it. -
_repair_orphan_tool_returnshistory processor auto-appended to everyBaseAgent's history-processor chain. Walks the post-compaction window, tracks the set of visibleToolCallPart.tool_call_ids, and strips anyToolReturnPartwhose id was sliced away by an earlier processor (keep_recent_messagesboundary cut,ContextManager._summarize_old_turnssummarisation of the matching call'sModelResponse, or any custom processor). Both OpenAI chat/completions and the Responses transport reject orphan tool returns with a 400 ("messages with role 'tool' must be a response to a preceeding message with 'tool_calls'"/"No tool call found for function call output with call_id ...") — these blew up mid-run on Polymath's first long Koopman session. Belt-and-suspenders: also applied insideContextManager.compact()'s own output. 0 backwards-compat hazards: stripping invalid parts only removes data that would have crashed the request anyway. Adjusted the 3_history_processors-length assertions intests/agents/test_base_agent.py,tests/test_budget_tool_results.py, andtests/test_context_manager.pyto account for the +1 processor; the test_emergency_preserves_min_tokens assertion shifted from min-recent-tokens to min-recent-turns since the repair sweep can trim orphan ToolReturnParts out of the recent window. -
MetricBundle.aggregate(list[MetricBundle])classmethod +n_trials/stdevfields — collapse N independent observations of the same candidate × example pair into one representative bundle with mean metrics + per-dimension standard deviation. New optional fieldsn_trials: int = 1andstdev: dict[str, float] | None = Noneare backward-compatible — existing single-observation bundles serialize and behave identically. Aggregate handles optional dimensions (confidence,robustness) gracefully: present-only mean,stdevonly when ≥2 trials supplied the dimension. Discovered while building the coding benchmark: single-trial LLM numbers swung ±10pp; multi-trial averaging is what stabilises an evaluator pipeline on weaker models. 10 new tests intests/optimization/test_bundle.py. -
Evaluator(n_trials_per_example: int = 1)parameter — when> 1, runs the agent N times per gold example and aggregates viaMetricBundle.aggregate. Each trial gets a fresh agent viaagent_factory(decoded)so trials are independent. Default1preserves the legacy single-shot behavior at no extra cost. When every trial of an example fails, the aggregate keeps the failure info (not a misleadingly-averaged 0.0 accuracy). Cost scales linearly withn_trials_per_example; useful for weaker models where evaluator variance is the dominant noise source. 6 new tests intests/optimization/test_evaluator.py. -
benchmarks/directory + first benchmark (benchmarks/coding/) — establishes the convention that every battery (or composition of batteries) ships with a reproducible head-to-head proving its value over a baseline.benchmarks/coding/run.pyis the entry point for the 10-problem coding benchmark behindnotebooks/12_combo_autonomous_coder.ipynb's +17pp pass@1 / +14pp test_pass_rate claim: one command reproduces the multi-trial average.benchmarks/README.mddocuments the convention for adding new benchmarks (require a baseline, default to multi-trial averaging, honest cost + variance disclosures);benchmarks/coding/README.mddocuments what's tested, how to reproduce, and the honest caveats (LLM variance, stronger-model headroom, latency cost). Promoted fromscratch/combo/after the iteration sprint surfaced that the numbers were buried where consumers couldn't find them. 4 new data-integrity tests intests/benchmarks/test_coding_fixture.pypin the fixture's 10-problem / 92-hidden-test shape so silent drift breaks CI. -
orqest.sandbox.run_in_sandbox(...)helper + non-raisingrun_in_sandbox_safe(...)sibling — collapse the canonical "run candidate Python in a sandbox, get the return value back" pattern from ~30 lines (buildGeneratedToolSpec→DynamicToolFactory.spawn()→ invoke → unwrapExecutionResult) into one async call.return_expressionparameter handles the common case of appendingreturn func(args)to a candidate function body. NewSandboxRunError(stage, code_snippet, underlying)exception captures validation vs execution failures with structured diagnostic context.run_in_sandbox_safereturns(success, output, error_msg)for callers that prefer inline failure handling over try/except. The lower-levelSandbox.execute()+GeneratedToolSpec+DynamicToolFactorypath stays the right choice when an agent itself needs to call a tool via pydantic-ai's tool-use mechanism; these helpers are for direct sandbox invocation from framework code. 13 new tests intests/sandbox/test_helpers.py. Discovered while buildingnotebooks/12_combo_autonomous_coder.ipynbwhere the evaluator wrapped this pattern ~30 lines per call. -
AgentSpecacceptsoutput_type: type[BaseModel]as an ergonomic alternative tooutput_schema: dict— declaring an agent's output shape with a Pydantic class is terser (and statically typed) compared to authoring a JSON Schema dict by hand. Both paths remain supported: a new@model_validatorenforces exactly-one-of so the two can't drift. The JSON Schema path stays the wire-format option (LLM-emittable specs, persistence); the Pydantic-class path is the in-process / code-side path.AgentFactory.spawn()dispatches on which is set with zero behavior change for existing callers. 6 new tests intests/autonomy/test_factory.py::TestAgentSpecOutputType. Discovered while buildingnotebooks/12_combo_autonomous_coder.ipynbwhere the JSON Schema dicts for two agents were ~16 lines vs ~4 lines per Pydantic class. -
BaseAgentvalidatesoutput_typeat construction — top-levelAny-typed Pydantic fields now raiseBaseAgentSchemaErrorimmediately at agent construction, with a message naming the offending field and concrete remediations (narrow the type, use a container ofAny, or use a JSON-string field). Previously this slipped through to first inference where OpenAI's structured-output endpoint surfaced an opaqueInvalid schema for function 'final_result'400 error with no breadcrumb. Containers holdingAny(list[Any],dict[str, Any]) and scalar output types (str,int, etc.) are still accepted — only top-levelAnyfields trigger rejection. Discovered while buildingnotebooks/12_combo_autonomous_coder.ipynbwhere this cost ~20 min of debugging on two separate occasions. 6 new tests intests/agents/test_base_agent.py::TestOutputTypeSchemaValidation. New exception classorqest.agents.base_agent.BaseAgentSchemaError(ValueError).
Docs¶
-
docs/concepts/sandbox.md— new "Recipe: test-driven loops over LLM-generated code" section. Documents the per-spec pattern for safely running LLM-generated candidate functions: bake source into a freshGeneratedToolSpecper (iteration × test), pass through the sandbox's static validator + RLIMIT-bounded subprocess, neverexec()strings against the parent namespace. Names what the pattern protects against and what it doesn't (kernel isolation, network egress, filesystem reads). Discovered while buildingnotebooks/12_combo_autonomous_coder.ipynbwhere the instinct was an exec-based test runner — which the validator rightly rejects. -
docs/concepts/topology_optimization.md— two new sections. (1) "Caveat: LLM-generated tests as evaluator feedback" — documents the Unsupervised Evaluation Paradox at the runtime layer: letting an LLM generate test labels for another LLM's code makes the loop worse on imperfect models, because hallucinated tests poison the fixer's signal (V2 of the coding benchmark regressed ~50pp from this exact pattern). Names what works instead (visible-test-only with keep-best). (2) "When NOT to useRuntimeTopologyDesigner— just hand-write a Pipeline" — honest framing on when the runtime designer's per-request LLM cost is worth paying vs. just instantiating known agents in known shapes. Three concrete tests for "should I reach for this primitive." -
docs/concepts/autonomy.md— new "When NOT to useMetaOrchestrator— just write the agents and a Pipeline" section, the parallel framing for the autonomy layer: the planner LLM call is the most expensive single invocation in the chain; only worth paying when the decomposition itself is the hard part. Cross-links to the topology version of the same principle. -
CLAUDE.md— new convention added to "Key Conventions": "Every battery ships with a benchmark." Namesbenchmarks/coding/as the canonical reference layout. Future batteries must ship with a reproducible head-to-head + README documenting expected numbers, variance, and cost.
Changed¶
RefinementLoopnow defaults tokeep_best=True— tracks the iteration with the highestEvalResult.scoreand returns that iteration's output if the final iteration regressed. Protects self-improving loops (test-driven coding, evaluator-feedback refinement) from "fixer breaks code that already worked" failure modes on imperfect models. Thepassed=Trueearly exit still returns the passing iteration's output (passing is the explicit success bar). When the evaluator never returns a numeric score, keep-best is a no-op and the legacy "return latest" behavior holds — so callers using boolean-only evaluators see zero behavior change.LoopResultgainsbest_iteration: int | Noneandbest_score: float | None(informational; populated regardless of the flag). Migration: setRefinementLoop(..., keep_best=False)to restore strict last-iteration semantics. Discovered while buildingnotebooks/12_combo_autonomous_coder.ipynb, where we hand-rolled this safety property and saw it lift combopass@1measurably ongpt-4o-mini. 6 new tests intests/orchestration/test_loop.py(full suite 1051 → 1057).
Added¶
- Tier-2 Docker sandbox + per-user persisted MCP tool library — closes the deferred W3.M item from
[0.3.0]. The host-sideorqest.sandbox.docker.DockerSandboxorchestrates per-session containers built from the neworqest/agent-runtimeimage (Dockerfileat repo root). The container ships its ownorqest.sandbox.docker_runtimepackage — a FastMCP server exposing four tools (execute_python,promote_tool,list_persisted_tools,forget_tool),SessionAuthMiddleware(HMAC-signed JWT with{sub: user_id, sid: session_id, exp}claims), per-agentuv venvs under/workspace/<session>/<agent_id>/venv(created in ~50 ms; cached per agent), and a SQLiteToolStoremounted as the per-user named volumeorqest-user-<user_id>. Promotion is threshold-based by default (N=3 successful invocations of the same(name, code_hash)auto-promote and firenotifications/tools/list_changed);eagerandoperator_approvalmodes also wired. Cross-session reuse: same user's NEXT session sees the persisted tools immediately ontools/list; cross-user isolation is enforced by per-user named volumes. Hardened defaults:--cap-drop=ALL --read-only --user 1000:1000 --pids-limit --memory --cpus,tinias PID 1, non-root user. Honest threat model: shared-kernel, protects against accidental damage and most prompt-injection scenarios; not adversarial-multi-tenant grade — Tier 3 (microvm) is the documented seam for that. orqest.sandbox.docker_runtime/— the in-container runtime package:server.build_server[_from_env],auth.SessionAuthMiddleware,executor.Executor(per-agent venv +uv pip installgated byORQEST_ALLOWED_PACKAGESallowlist + AST validator + RLIMIT-bounded subprocess),store.ToolStore(per-user SQLite,(name, version)PK, deduplication byimplementation_hash).orqest.sandbox.docker.DockerSandbox(host-side) — Sandbox-Protocol-conformant; lifecycle is an async context manager (__aenter__→docker run+ MCP boot-wait + JWT-auth open;__aexit__→docker rm -f, named volume persists). Requireduser_id+session_idctor args (framework-issued; the LLM never sees them). Mints a fresh HMAC secret per construction by default; passes it to the container viaORQEST_HMAC_SECRET.orqest.sandbox.jwt— minimal HS256 JWT (encode/decode/verify), constant-time signature compare,alg=noneconfusion attack defended. Avoids pulling inpyjwtfor this single internal use.orqest.sandbox._compat— soft-import boundary for the optionaldockerSDK; friendlyImportErrorwith the install hint at first call todocker_from_env()rather than at module load.orqest.workbench.Workbench(user_id=..., session_id=...)— required ctor args (BREAKING — see below). Newwith_docker_sandbox(*, user_id, session_id, image, allowed_packages, ...)factory mirrorswith_healing(...); lazy-importsorqest.sandbox.dockerso the workbench stays import-light.orqest.autonomy.GeneratedToolSpec.dependencies: list[str]— additive; LLM can declare pip dependencies that the container's executor will install into the agent's venv (gated byallowed_packages).orqest.autonomy.tool_factory.spawn(spec, *, agent_id=None)andAgentFactory._resolve_tools(..., *, agent_id=None)—agent_idpropagates through the factory chain toSandbox.execute(..., agent_id=...). Tier-0/1 sandboxes accept-and-ignore for backward compat; Tier-2 routes to per-agent venvs.orqest.sandbox.protocol.Sandbox.execute(...)— additiveagent_id: str | None = Noneanddependencies: list[str] | None = Nonekwargs.orqest.mcp.MCPServerConfig.headers: dict[str, str]andtransport: Literal["stdio","sse","streamable-http"]— additive.MCPConnection._open_transportlearns astreamable-httpbranch viamcp.client.streamable_http.streamablehttp_client, forwardingconfig.headersfor theAuthorization: Bearer ...workflow.orqest.memory.MemoryEntry.memory_type = Literal["semantic","episodic","procedural","tool"](extension);MemoryFiltermirrored.orqest.memory.strategies.ToolStrategy(exact-name match with FTS5 fallback).orqest.memory.MemoryConfig.tool: PerKindConfig(versioning enabled; no TTL by default).- ~74 new tests across
tests/sandbox/(test_jwt.py,test_docker_compat.py,test_docker.pymarkeddocker),tests/sandbox/docker_runtime/(test_store.py,test_executor.py,test_auth.py,test_server.py),tests/mcp/test_streamable_http_transport.py,tests/memory/test_tool_memory_type.py,tests/autonomy/test_generated_tool_spec_dependencies.py,tests/workbench/test_user_session.py. Full suite: 1051 passing (no daemon) + 13 markeddocker(requiredockerdaemon AND theorqest/agent-runtimeimage — green when both available). New pytest markerdockerregistered in[tool.pytest.ini_options]. - New optional dependency group
[dependency-groups] docker = ["docker>=7.0", "httpx>=0.27"]. Mirrors theoptimizationpattern. Soft-imported viaorqest.sandbox._compat. BaseAgent(reasoning=...)— a provider-agnostic reasoning/thinking knob ("minimal"|"low"|"medium"|"high"). pydantic-ai exposes thinking through a differentModelSettingskey per provider (anthropic_thinking/openai_reasoning_effort/google_thinking_config/openrouter_reasoning);reasoningcollapses that into one effort level, translated and merged intomodel_settings— explicitmodel_settingskeys win on conflict. For budget-based providers (Anthropic, Google) a sensiblemax_tokensis filled when unset so reasoning works out of the box. Neworqest.utils.reasoningmodule (ReasoningEffort,resolve_reasoning_settings);ReasoningEffortis re-exported fromorqest.agents.- New battery
orqest.optimization— reflective prompt evolution via GEPA (Genetic-Pareto, Agrawal et al., ICLR 2026 Oral). Three-layer split mirroringhealing/: encoding (Genome+PromptGene/ScalarGene/CategoricalGene), evaluation (Evaluator+GoldExample+MetricBundle/MetricWeightsaggregating accuracy / confidence / cost / latency / robustness), adaptation (OrqestGEPAAdapter+OrqestEvalBatch+OptimizationRunner+OptimizationResult+apply_result/OptimizationDiff). The adapter feeds GEPA both per-example scalarscoresand per-objectiveobjective_scoresso the nativefrontier_type="hybrid"Pareto frontier discovers tradeoffs across both axes.apply_result(dry_run=True)is the default; commit invalidates the cachedpydantic_ai.Agentso the new prompt actually takes effect. Async-bridge handles both fresh-process (asyncio.run) and Jupyter (worker thread). 62 new tests (62/62 green; full suite 689 → 751). Optional dependency —uv sync --group optimization. Concept doc atdocs/concepts/optimization.md; demo notebooksnotebooks/06_optimization_basic.ipynb(research summarizer) andnotebooks/07_optimization_compound.ipynb(planner insideMetaOrchestrator). Scalar/categorical gene evolution gated byOptimizationConfig.enable_scalar_genes/enable_categorical_genesuntil the upstream W1.1 wiring lands. - Topology evolution under
orqest.optimization(W3) — ADAS-style structural search over a typedTopologySpecIR. Two new sub-batteries:- Orchestration spec layer (
orqest/orchestration/spec.py+hydrate.py) — Pydantic models forPipelineSpec/ParallelSpec/RouterSpec/RefinementLoopSpec/AgentStepSpec/FunctionStepSpecplus the top-levelTopologySpecandOperationSpecdiscriminated unions.topology_from_spec()hydrates a spec into a live runtime via an explicitCallableRegistry+ agent factory map — noeval, noexec, no name forgery. Closes the audit-named gap (LLM cannot emit composition topology at runtime). Independently useful regardless of search. - Topology search engine (
orqest/optimization/topology.py+meta_agent.py) —TopologyGene(Pydantic gene whose value is a serializedTopologySpec),TopologyEvaluator(subclass ofEvaluator; addsn_agents/depthtoMetricBundle.raw), andMetaAgentSearch(the ADAS-style design → reflexion → evaluate → archive loop). The meta agent emits typed JSON, not raw Python — sidestepping ADAS'sexec()-in-process gap and the entire sandbox problem. Pluggable archive strategies (top_kdefault,cumulativeADAS-faithful,parallelper the 2510.06711 critique). Pydantic ValidationError + hydration KeyError feed back to the meta agent as debug-retry feedback (analogue of ADAS's traceback retry, but type-safe). Returns anOptimizationResultshaped identically to the GEPA path soapply_resultand downstream consumers work without dispatch. Two-phase composition with GEPA is the recommended path (ADAS first with fixed strong prompts, GEPA on the winner) — never nest, the multiplicative cost is fatal. Concept doc atdocs/concepts/topology_optimization.md; demo notebooksnotebooks/08_topology_search_basic.ipynbandnotebooks/09_topology_with_gepa.ipynb. 95 new tests (44 spec/hydrate + 20 topology + 25 meta_agent + 6 topology-apply; full suite 768 → 863).
- Orchestration spec layer (
MetaAgentConfigre-exported fromorqestroot (matchesHealingConfig/MetacognitionConfig/OptimizationConfigpattern).apply_resultdiff renderer now JSON pretty-prints Pydantic-model gene values (typedTopologySpecslots get readable line-by-line diffs).- Runtime topology design — the runtime planner sibling to
MetaOrchestrator. Both new modules live underorqest.autonomy/(notorqest.optimization/) because they are runtime planners, not optimizers — there's no loss function, no per-request scoring, no Pareto archive. They share theTopologySpecIR withMetaAgentSearchbecause the IR is provenance-agnostic, but the relationship is shared infrastructure, not shared algorithm:orqest.autonomy.runtime—RuntimeTopologyDesigner(per-requestTopologySpecsynthesis via a user-providedBaseAgent[GlobalState, TopologyDesign]), plus theTopologyCacheProtocol with three implementations:NoCache(default, zero state),InMemoryLRU(exact-match goal string), andMemoryStoreCache(backed byLocalMemoryStorewith semantic recall + reliability decay on execution failure). The cache usesmemory_type="semantic"with a namespacedsource_agent="topology_cache"slot; reliability decay reuses the existingPerKindConfig.decay_on_failuremachinery.verify_on_hit=Trueby default catches stale agent / callable references after registry changes; optionalfallback_specreturns a safe minimal topology when design fails. Seed-library bootstrap accepts a list of validated topologies (typically the Pareto front from an offlineMetaAgentSearchrun) and biases the designer toward known-good shapes. Honest framing: the cache decays only ontopology.runexceptions, not on bad-quality outputs — that's W3.E (deferred).orqest.autonomy.topology_orchestrator—TopologyOrchestratoris the topology-shaped sibling toMetaOrchestrator. Per-request loop: design (cache or fresh) → hydrate → run → record outcome. Returns a typedTopologyExecutionResultwith structural metrics (n_agents,depth), timing breakdown (design_ms,execution_ms,total_ms), andcache_hitsignal. Bus events:topology.execution_completed/topology.execution_failed/topology.cache_hit/topology.cache_miss/topology.designed/topology.design_failed/topology.fallback_used.- Import paths — both modules are imported via the explicit submodule path:
from orqest.autonomy.runtime import RuntimeTopologyDesigner, MemoryStoreCache, NoCache, InMemoryLRU, TopologyCachefrom orqest.autonomy.topology_orchestrator import TopologyOrchestrator, TopologyExecutionResultNot re-exported fromorqest.autonomy.__init__to avoid a circular import (eagerly loading them at autonomy package init triggers loadingoptimization.meta_agentmid-init viatopology→autonomy.factory). Matches the pattern used by other heavy batteries (e.g.,from orqest.compound import SubAgentTool).
- 35 new tests (25 runtime + 10 orchestrator; full suite 863 → 898). Concept doc section "Runtime topology design" appended to
docs/concepts/topology_optimization.md. Demo notebooknotebooks/10_runtime_topology.ipynbwalks through cold design, cache reuse, full orchestrator loop, seed-library bootstrap, and online-learning failure decay end-to-end.
unpack_topology_outputpromoted from private_unpack_topology_outputinorqest.optimization.topology— used by bothTopologyEvaluatorandTopologyOrchestratorto extract the meaningful payload (ParallelResult.merged/LoopResult.output) from a topology's.run()result.orqest.sandboxsubpackage — safe execution surface for LLM-generated Python (closes the Phase-3 deferredToolSandboxitem). Two-stage Protocol:validate(code, allowed_imports)(static AST checks; raisesValidationErroron disallowed import / forbidden call / dunder access / syntax error) andexecute(code, args, allowed_imports, timeout_s, memory_mb)(always returnsExecutionResult; never raises for user-code failures). Two backends ship:InProcessSandbox(unsafe=True)(Tier 0 —exec()in a restricted namespace; refuses to construct withoutunsafe=Truebecause there is no real isolation) andSubprocessSandbox(Tier 1, production default — freshpython -csubprocess per call withRLIMIT_AS+RLIMIT_CPU(POSIX) + outerasyncio.wait_fortimeout; JSON args/result via stdin/stdout). Default-deny imports — emptyallowed_importsrejects anyimportstatement. Defense-in-depth — the subprocess re-validates internally before execution. Concept doc atdocs/concepts/sandbox.md. No new dependencies (subprocess / resource / ast / json are stdlib).- Dynamic tool spawning — closes the autonomy ladder's missing rung.
AgentSpec.toolspreviously dropped anyToolSpecwhose name wasn't pre-registered; now an LLM-emittedGeneratedToolSpec(carryingimplementation: str+allowed_imports: set[str]+timeout_s+memory_mb) is materialized at runtime viaDynamicToolFactory(sandbox=...). Three new pieces:orqest.autonomy.spec.GeneratedToolSpec— Pydantic model carrying the implementation.AgentSpec.toolswidened tolist[ToolSpec | GeneratedToolSpec]; Pydantic v2 smart-union dispatches by structure (theimplementationfield is the disambiguator). Backward-compatible — existingToolSpec(name=...)construction unchanged.orqest.autonomy.tool_factory.DynamicToolFactory— wraps aSandboxto produce runnablepydantic_ai.Toolobjects fromGeneratedToolSpec. Bus events:tool.spawned/tool.spawn_failed/sandbox.validation_rejected/tool.invocation_completed/tool.invocation_failed. The spawnedToolreturns a structured error dict on sandbox failure (not a Python exception) so the agent loop sees it as a tool result.orqest.autonomy.AgentFactory(tool_factory=...)—_resolve_toolslearns to dispatch onisinstance:ToolSpec→ registry lookup (existing path),GeneratedToolSpec→tool_factory.spawn(spec). Whentool_factory is Noneand aGeneratedToolSpecappears, log + skip (matches the existing graceful-skip behavior for unknown registry names). Internal_spawn_generatedasync-bridges viaasyncio.run(no loop) or worker thread (loop already running) — same pattern asOrqestGEPAAdapter._run_async.BaseAgent.add_tool(tool)— appends toself.toolsand invalidates the cachedpydantic_ai.Agentso the next access rebuilds with the new tool list. Idempotent for tools sharing aname(last-write-wins). Closes the "agent encounters a capability gap mid-run; orchestrator hands it the missing tool" path.orqest.autonomy.__init__re-exportsGeneratedToolSpec+DynamicToolFactory.- 61 new tests (34 sandbox + 14 tool_factory + 7 factory-with-generated-tools + 6 add_tool; full suite 898 → 959). Concept docs: new
docs/concepts/sandbox.md; appended "Dynamic tool spawning" section todocs/concepts/autonomy.md. Demo notebooknotebooks/11_dynamic_tools.ipynbwalks through sandbox standalone, factory spawn, mixed-tools dispatch, runtime add_tool, and an end-to-end real-LLM run where the agent uses a tool that didn't exist at agent-construction time. - Future seams (deferred): W3.J — procedural-memory persistence for spawned tools (
Skillshape); W3.K —SubprocessPoolSandboxfor amortizing startup cost; W3.L —E2BSandbox(hosted micro-VM, optional dep); W3.M —DockerSandbox/FirecrackerSandbox(real network/filesystem isolation); W3.C revisited — ADAS sandboxed codegen now unblocked.
Breaking Changes¶
Workbench(...)now requiresuser_id: strandsession_id: strkeyword args. Existing in-tree consumers (notebooks, examples, tests) updated. The args are framework-issued — never derived from LLM-visible context — and become the strict isolation key for the Docker tier's per-user persisted tool library. The previousWorkbench()no-arg form was used by ~20 in-tree call sites; all migrated. No external consumers exist (orqest is unpublished as of0.7.0).
Dependencies¶
- New optional dependency group
[dependency-groups] optimization = ["gepa>=0.1.1"]. Not in core; pulls litellm + datasets + tiktoken transitively (~50–80 MB). Soft-imported viaorqest.optimization._compatwith a friendlyImportErrorwhen absent. - New optional dependency group
[dependency-groups] docker = ["docker>=7.0", "httpx>=0.27"]. Tier-2 host-side only; theorqest/agent-runtimecontainer ships the rest. Soft-imported viaorqest.sandbox._compat. - Topology evolution adds no new dependencies — uses the existing pydantic-ai meta-agent + the orchestration primitives already in core.
[0.4.0] - 2026-05-14¶
The advance pass — "complete & honest." Finishes the [0.3.0] preview tier into Tier 1: every capability that was a designed-but-unwired seam is now wired and test-covered. Test suite: 664 → 670.
Added¶
PerKindConfig.ttl_days+LocalMemoryStore.prune_expired()— per-kind retention windows; a best-effort maintenance call deletes entries past their TTL and returns the count. A kind withttl_days=Noneis never pruned.PerKindConfig.version_on_edit— when enabled, re-storing a procedural skill by name bumps itsversionone past the highest stored version and keeps the prior rows as an audit trail.LocalMemoryStore(embedder=...)— a pluggable sync-or-async embedder. When set,store()embeds entry content andSemanticStrategyranks recall by cosine similarity over stored vectors; FTS5 / LIKE otherwise. Newdefault_strategy_table(embedder=...)parameter andembed_text/_cosinehelpers inorqest.memory.strategies.MCPDiscovery(well_known_urls=...)—search()probes the configured base URLs for/.well-known/mcp.json(highest intent, tried first), then queries the registry endpoints, deduplicating by name. The documented well-known-manifest discovery source is now real.
Changed¶
- The
RecoveryActionunion stays lean (AbortRun|EscalateToUser) by design — model-level recovery isFallbackModel, tool-level recovery isDiscoveryHook. The advance pass evaluated re-introducingRetryDifferentModel/DiscoverAndRetry/RetrySameTooland concluded they duplicated those dedicated, composable mechanisms.
Preview¶
Still accepted but not yet wired:
MemoryConfig.backend="supabase"/supabase_*— the pgvector backend.MCPDiscoveryregistry response-shape parsing — untested against live registries; no web-search fallback.
[0.3.0] - 2026-05-14¶
The reconcile pass. A contradiction audit found ~27 places where the code and the docs disagreed — most often an "intent layer" (configs, types, hooks) built ahead of the "effect layer" that consumes it. This release wires the cheap gaps, deletes the dead ones, and corrects every stale claim. Tiered honesty contract: each subsystem's primary path is functional end-to-end; specific advanced capabilities are explicitly labelled Preview. Test suite: 655 → 664.
Breaking Changes¶
compound.sub_agent_tool.EvalResultrenamed toSubAgentEvalResult— it collided by name withorchestration.loop.EvalResult(a different shape).RecoveryActionunion reduced toEscalateToUser | AbortRun.RetrySameTool/RetryDifferentModel/DiscoverAndRetryare removed — they produced payloads no compound flow consumed.PerKindConfigreduced todecay_on_failure+prune_below;ttl_daysandversion_on_editremoved (no maintenance routine / versioning logic ever existed).LocalMemoryStore.__init__—db_pathis now optional and aconfig: MemoryConfigkeyword param was added.- Prompt delivery —
CompoundTool.runandSubAgentTool.runnow inject the prompt intostateas a user message.CompoundToolpreviously never passed the prompt to the agent at all;SubAgentToolpassed it only as anote=kwarg (still forwarded for agents that read it). RefinementLoop.__init__raisesValueErrorwhenagent_self_evalis set but the agent has noconfidence_protocol.- Default model is
openai:gpt-4.1everywhere (wasopenai:gpt-3.5-turboin config,openai:gpt-4oin autonomy / the MCP server). - Packaging — the explicit
openaidependency is removed (still bundled transitively bypydantic-ai);pytest, the duplicatedotenv, and the unusedmarkdowndeps are removed.
Fixed¶
on_errorhook decisions are now consumed —CompoundToolandMetaOrchestratorhonor aRedirectfromon_erroras a bounded single retry.DiscoveryHook(whose only behaviour is anon_errorRedirect) is functional again.PerKindConfig.decay_on_failure/prune_beloware now wired intoLocalMemoryStore.update_reliability(the decay factor and prune floor were hardcoded; the config was inert).MCPConnection.connect()branches onconfig.transport—sseservers (including everythingMCPDiscoveryreturns) can now actually connect instead of being driven through the stdio client with an empty command.MetaOrchestratormemory-reuse is keyed consistently onsubtask.name— the recall query, skill trigger, and spawned-agent cache were mismatched, so reuse was effectively never hit.RefinementLoopno longer silently accepts anagent_self_evalagent with no confidence protocol (the loop could never exit via"confident").- ~12 stale doc / docstring claims corrected: the
salience"side-table cache", theWatchdogHook"hook.shadowed event",MCPDiscovery's "three discovery sources", theContextManagersalience threshold, generative UI's "5 first-party components" (it is 17 across 3 layers), the MetaOrchestrator "successful specs" persistence claim, the "pay for what you use" provider claim,load_sys_prompttyping, and missing__all__exports (WebSearchResponse,budget_tool_results).
Removed¶
RetrySameTool,RetryDifferentModel,DiscoverAndRetryrecovery actions and thehealing.retry_initiatedevent.HealingConfig.abort_on_unresolved_loop— dead config flag, never read.MCPConfig.connection_timeout— dead config field, never read.metacognition.protocol._SelfRating— dead class, never referenced.PerKindConfig.ttl_days/version_on_editand the_episodic_default/_procedural_defaultfactories.
Preview¶
The following are accepted but not yet wired end-to-end, and are now labelled as such in their docstrings and concept docs:
MemoryConfig.backend/supabase_*/embedding_*— designed seams for a future pgvector backend and embedding-based retrieval.MCPDiscoveryonline registry search — untested against live registries;probe_wellknownexists but is not wired intosearch().
[0.2.0] - 2026-04-25¶
The cognitive-substrate completion. Three implementation waves landed in sequence on the same day (Wave 1: HookDecision + procedural memory + metacognition; Wave 2: healing + MCP auto-wire; Wave 3: generative UI). Test suite: 360 → 612 (+252).
Added¶
orqest.metacognition (Wave 1.3 — vision feature #3 "Metacognition primitives"):
- EnrichedOutput[OutputT] — Pydantic generic pairing an output with confidence (float | None in [0, 1]), uncertainty_targets: list[str], capability_boundary: bool, protocol_name, and free-form metadata
- ConfidenceProtocol Protocol + three concrete strategies: StructuredOutputProtocol (zero-cost; lifts self_confidence/uncertain_about/outside_my_capability off the agent's own OutputT), LLMSelfRatingProtocol (+1 LLM call; rater agent emits JSON, markdown-fence-tolerant parser), EnsembleProtocol(k=N) (+k–1 parallel calls; pairwise-agreement confidence)
- MetacognitionHook — ToolHook that emits metacognition.confidence events whenever a tool result is an EnrichedOutput
- MetacognitionConfig frozen dataclass with redecompose_threshold / max_redecompositions / confidence_floor
- confidence_salience / recency_salience — pure salience scorers for ContextManager integration
- BaseAgent.run_enriched(state, *, confidence_protocol=None) -> EnrichedOutput[OutputT] (additive; run untouched)
- BaseAgent ctor gains keyword-only confidence_protocol for an agent-level default
- RefinementLoop ctor gains confidence_threshold: float | None (new exit reason "confident") and agent_self_eval: BaseAgent | None
- SubAgentResult.confidence / uncertainty_targets / capability_boundary (additive optional fields); SubAgentTool.run(use_enriched=True) lifts the final-iteration enrichment
- ContextManager(salience_fn=...) — pluggable per-message salience scorer; emergency truncation rescues high-salience old messages
- MetaOrchestrator(metacognition: MetacognitionConfig | None = None) — re-decomposes remaining subtasks when _extract_confidence(result.output) < redecompose_threshold
orqest.healing (Wave 2 — vision feature #4 "Self-healing primitives"):
- HookDecision discriminated union: Continue / Skip(reason, stub_result) / Redirect(new_args, new_tool, reason) / Abort(reason) (Wave 1.1 — also a Wave 2 prerequisite)
- HookAbortError — propagated when a hook returns Abort
- ToolHook protocol upgrade: methods may return HookDecision | None. HookRunner._safe_call auto-wraps None → Continue. HookRunner._aggregate first-non-Continue-wins with Abort short-circuit (Wave 1.1)
- CompoundTool.run, run_with_retry, MetaOrchestrator._execute_subtask honor Skip / Redirect / Abort (Wave 1.1)
- Watchdog Protocol + Detection Pydantic model (Wave 2.C)
- StallDetector (timeout on open tool calls, idempotent subscribe), LoopDetector (sliding window of (tool_name, args_hash)), RegressionDetector (subscribes to metacognition.confidence events; graceful no-op without metacog) (Wave 2.C)
- RecoveryAction discriminated union: RetrySameTool / RetryDifferentModel / EscalateToUser / AbortRun / DiscoverAndRetry + default_policy (Wave 2.C)
- WatchdogHook — ToolHook mapping Detection → policy → HookDecision. Emits healing.action events (Wave 2.C)
- FallbackModel — subclasses pydantic_ai.models.Model; sticky failover; transient classifier (5xx/timeout → fall back; auth/validation → propagate); emits healing.model_fallback (Wave 2.C)
- resolve_model_with_fallback(models, *, api_key, bus, transient_predicate) — accepts a chain; per-provider key map with graceful skip on missing keys (Wave 2.C)
- HealingRunner async context manager — wires watchdogs to a bus, runs poll loop, emits healing.detection events, owns the WatchdogHook and (optional) FallbackModel (Wave 2.C)
- Workbench.with_healing(config, *, api_key=...) convenience factory (lazy import) (Wave 2.C)
- HealingConfig frozen dataclass (Wave 2.C)
- ToolRegistry.get_or_discover(name, *, discovery, manager, permission, audit_bus, max_servers) — deliberate auto-discovery path (Wave 2.D)
- DiscoveryHook — ToolHook recovering from "tool not found" runtime errors via MCP discovery; returns Redirect(new_tool=name) after registration (Wave 2.D)
- PermissionGate Protocol + AllowAll / DenyAll (default) / AllowList (regex) (Wave 2.D)
- Audit-log events: discovery.requested / discovery.connected / discovery.denied / discovery.failed (Wave 2.D)
orqest.ui (Wave 3 — vision feature #5 "Generative UI"):
- UIComponentSpec[T] — generic Pydantic with component_type Literal discriminator, component_id, typed data: T, metadata, created_at
- UIDeltaEvent — partial update with op: Literal["replace","merge","append","remove"] + dot-path + value
- UIDeltaOp type alias
- ComponentRegistry per-Workbench (no module singleton): register, get, list_types, validate_payload
- default_registry() — pre-loads first-party components
- UIEmitter(bus) — init / delta / remove convenience over EventBus
- ui_init_event_type / ui_delta_event_type / ui_remove_event_type helpers (event-type convention ui.<component_type>.{init,delta,remove})
- First-party components: PlanComponent, ChartComponent (line/bar/scatter/pie/heatmap with typed ChartSeries), TableComponent (typed TableColumn), FormComponent (typed FormField), TakeoverDialogComponent (confirm/input/choice), plus declarative grammars (VegaChartComponent, MermaidComponent, LatexComponent, JsonViewerComponent) and the SandboxedHTMLComponent escape hatch
- ExecutionPlan.enable_ui_events(*, component_id="plan") — opt-in flag-gated dual emission of ui.plan.init/ui.plan.delta alongside legacy plan.init/plan.task.updated
- ExecutionPlan.as_component() — wraps the plan as a PlanComponent
- Workbench(ui_registry=..., auto_register_first_party_ui=True) ctor kwargs
Cognitive memory typology (Wave 1.2 — vision feature #2 "Cognitive memory typology" completion):
- MemoryEntry.memory_type extended to Literal["semantic", "episodic", "procedural"]
- Skill / ToolCallSpec / SkillExample Pydantic shapes (procedural payload in MemoryEntry.structured_content)
- MemoryEntry.structured_content: dict[str, Any] | None (validation gated to memory_type == "procedural")
- MemoryFilter.skill_name / skill_min_version for procedural filtering
- RetrievalStrategy Protocol + SemanticStrategy (legacy FTS5/LIKE behavior preserved) / EpisodicStrategy (ORDER BY created_at DESC) / ProceduralStrategy (exact trigger match + optional injected fuzzy judge)
- default_strategy_table() — the per-kind dispatch table consumed by LocalMemoryStore
- LocalMemoryStore(strategies=...) — strategy override for custom backends
- Best-effort ALTER TABLE migration for the structured_content column
- MemoryConfig extended with semantic / episodic / procedural PerKindConfig fields (TTL / decay / version-on-edit)
- MetaOrchestrator._find_or_spawn dual-write migration: persists both episodic mirror (legacy) and procedural Skill entries; recall is procedural-first with episodic fallback
Test suite: 360 → 612 at wave-3 ship (+252 across the three waves; 360 pre-existing tests stayed green at every wave boundary). Subsequent consumer-side polish (2026-04-26) brought the suite to 655.
[0.1.0] - 2026-04-24¶
Phases 2–5 of the original Orqest roadmap stabilized: Memory, Autonomy, Observability, and MCP. The substrate that the Wave 1–3 cognitive features in 0.2.0 build on.
Added¶
orqest.memory (Phase 2):
- MemoryStore Protocol — store, recall, forget, update_reliability, count
- MemoryEntry — content + memory_type (initially "semantic" | "episodic") + source agent + confidence + reliability score + metadata + access tracking
- MemoryFilter — query constraints (memory_type, source_agent, min_confidence, before/after timestamps)
- LocalMemoryStore — SQLite + FTS5 backend (with LIKE fallback when FTS5 unavailable). Lazy init; best-effort error handling (logged, never raised). Self-healing reliability decay on failure.
- MemoryConfig — frozen dataclass for backend selection and embedding model
orqest.autonomy (Phase 3):
- AgentSpec / ToolSpec — serializable contracts; LLM can emit these as structured output
- AgentFactory.spawn(spec) -> DynamicAgent — builds a Pydantic output model from JSON Schema via pydantic.create_model(); resolves tools from the registry; injects constraints into the system prompt
- ToolRegistry — register, get, search(query, k) (keyword scoring), list_all, remove, dunder methods
- MetaOrchestrator(planner_agent, registry, default_model) — goal → TaskDecomposition → per SubTask spawn-or-find agent → execute → SubTaskResult → aggregated ExecutionResult
- DynamicAgent extends BaseAgent[GlobalState, BaseModel]
orqest.observability (Phase 4):
- Span — trace_id, span_id, parent_span_id, name, agent_name, timing, status, attributes, events
- Tracer Protocol; JSONTracer is the default in-memory implementation (no external deps)
- AgentEvent — frozen immutable event (event_type, agent_name, timestamp, data, span_id, trace_id)
- EventBus — in-process pub/sub; subscribe(event_type), subscribe_all, emit(event). Handler exceptions logged and discarded (fire-and-forget)
- EventBusPublishHook — bridges ToolHook → EventBus; emits tool.before, tool.after, tool.error with configurable preview truncation
- sse_sidecar(bus, replay=(), heartbeat_s=15.0, queue_size=256) — async iterator yielding SSE-formatted strings; ring-buffered against slow consumers; optional historical replay for reconnection
orqest.workbench:
- Workbench — runtime container bundling memory + tracer + event_bus + recent_events ring buffer
orqest.compound:
- SubAgentTool[StateT, ResultT] + SubAgentResult — captures agent → executor → state-update → optional refinement; refinement-loop integrated
orqest.plan:
- ExecutionPlan, PlanTask, PlanSubtask, PlanStatus — typed multi-step workflow tracking; to_sse_init() is byte-stable as a frontend contract
orqest.mcp (Phase 5):
- MCPServerConfig / MCPConfig — explicit server definitions + auto-discovery toggle
- MCPConnection(config) — single-server lifecycle: await connect() → .tools → await disconnect()
- MCPServerManager — multi-server orchestration, async with context manager, connect_all, get_all_tools (flat list), search_tools
- MCPToolAdapter.adapt[_many] — MCP tool definitions → pydantic-ai Tool instances (graceful error-string return wrapper)
- MCPDiscovery.search(query, max_results) — online discovery (registry + well-known manifests + web fallback)
- Auto-discovery scans ~/.claude.json, ~/.claude/claude.json, ~/.config/Claude/claude_desktop_config.json
- create_orqest_server(factory, registry, meta, default_model, api_key) — FastMCP server exposing create_agent, run_agent, solve_goal, list_agents. Run with python -m orqest.mcp.server
orqest.tools.web:
- web_search(query, k, provider, api_key) — pluggable provider strategy (tavily / exa / brave / serper); graceful degradation when key missing
- web_fetch(url) — plain GET with WebFetchResult(url, status_code, content_type, text, truncated)
Multi-modal prompts and streaming:
- BaseAgent.call_model / call_model_stream / stream_output / stream_events accept str | Sequence[UserContent] (images, PDFs, audio, video via pydantic-ai's ImageUrl / DocumentUrl / AudioUrl / VideoUrl / BinaryContent)
- Prompt type alias (str | Sequence[UserContent]) exported from orqest.agents
- call_model_stream() — async context manager for streaming with history wiring
- stream_output() — async generator yielding partial structured output as the LLM generates tokens
- stream_events() — async generator yielding all agent events including tool call/result visibility
Composition extensions:
- as_tool() — wrap any BaseAgent as a pydantic-ai Tool for stateless orchestrator invocation
- call_model() on BaseAgent — multi-turn conversation support with automatic history wiring
- CompoundTool pattern — agent → executor → state update with HookRunner dispatch
- run_with_retry() — exception-based retry with default enrichment
- ContextManager — token-aware three-tier compaction (tool-result snip → turn summarization at 60% → emergency truncation at 85%)
- Documentation site (MkDocs Material) with concept docs + auto-generated API reference
Changed¶
GlobalState.message_historytyped aslist[ModelMessage]instead oflist[Any]
[0.0.1] - 2025-07-21¶
Added¶
BaseAgent[StateT, OutputT]— generic, async-first abstract base class for agentsGlobalState— conversation state with app-level messages and pydantic-ai message historykeep_recent_messages()— history truncation preserving first message and turn integrityresolve_model()— multi-provider model routing (OpenAI, Anthropic, Google, OpenRouter) usingprovider:model_idformatOrqestConfig— frozen dataclass for runtime configurationload_config()andget_default_config()— explicit config loading with no import-time side effectsload_sys_prompt()— system prompt file loader with upward directory search- Tool and toolset registration on
BaseAgent - Custom history processor support
- Example notebook
01_basic_agentwith single agent and structured output - Test suite covering all modules