Skip to content

Runbook: Orchestrator Engine (Phase 3)

Operating agents, tools, the supervisor, the API, and Langfuse tracing.

Prerequisites

docker compose up -d        # incl. langfuse (+ langfuse-db)
ollama serve                # llama3.2:3b + nomic-embed-text
aop diagnose                # langfuse row should be OK

Ingest a corpus so retrieval tools have data:

aop ingest tests\fixtures\rag.md -c default

Agents & tools

aop agents                                  # capabilities + tags
aop tools                                   # MCP tool manifests
aop agent run "Explain re-ranking, then critique it." --steps
aop agent run "What is a vector database?" --agent react
aop agent run "..." --namespace agent7      # scoped tools/memory

Agents: react, plan_execute, reflexion, critic. Omit --agent to let the supervisor route by capability.

API

aop serve                                   # http://127.0.0.1:8000
# REST
curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/agents
curl -X POST http://127.0.0.1:8000/agent/run -H "content-type: application/json" -d '{"task":"hello","agent":"react"}'
# OpenAPI docs at /docs

WebSocket /ws/agent: send {"task": "...", "agent": "react"}; receive {"type":"start"}, a stream of {"type":"step","step":{...}}, then {"type":"final","result":{...}}.

Observability (Langfuse)

  • UI: http://localhost:3001 (login dev@aop.local / aop-dev-password).
  • Traces appear under project AOP after each agent run / /agent/run.
  • Tracing is best-effort: if Langfuse is down, runs still succeed (steps go to structlog) and trace_id is None.

Troubleshooting

Symptom Cause Fix
Langfuse 500 / "Invalid environment variables" bad ENCRYPTION_KEY (must be 64 hex) use a valid key; docker compose up -d langfuse --force-recreate.
trace_id is null Langfuse unreachable/misconfigured check aop diagnose; verify keys; non-fatal.
agent ignores tools small model didn't emit an Action expected with llama3.2:3b; use a larger/cloud model for robust tool use.
/agent/run 500 Ollama down ollama serve.

Tuning

AOP_AGENT_MAX_ITERS, AOP_AGENT_TEMPERATURE, AOP_API_HOST/PORT, AOP_LANGFUSE_*.