AOP Guidebook — Start Here & Explore Everything

A hands-on tour of the Agentic Orchestrator Platform. Follow it top-to-bottom the first time; afterwards jump to any section. Commands are PowerShell (Windows); POSIX equivalents are noted where they differ.

Mental model: one Python backend (aop, a CLI and a FastAPI service) + one Next.js frontend (Canvas + Fine-Tuning pages) + a Docker data stack + local Ollama models. Local-first, no mocks — if a service is down, commands say so with a fix.

---

0. Prerequisites (one-time)

Need	Why	Check
Ollama running	local LLM (chat + embeddings)	ollama --version
Docker Desktop	data stack (Qdrant, Redis, Postgres, Redpanda, Mongo, Langfuse)	docker compose version
Python 3.12–3.14	backend	py --version
Node 20+ & pnpm 10	frontend	pnpm --version
git	project-import from URLs	git --version

Models — two kinds

1. Ollama models (chat + embeddings) — you pull these manually, once. These power everything by default and run fully locally via the Ollama daemon.

ollama pull llama3.2:3b        # default CHAT model (~2 GB) — agents, RAG answers, summaries
ollama pull nomic-embed-text   # default EMBEDDINGS model (~274 MB, 768-dim) — indexing & search
ollama list                    # verify both appear

The Ollama daemon must be running (Windows installs it as a background service automatically; otherwise run ollama serve). The platform talks to it at http://localhost:11434 (AOP_OLLAMA_BASE_URL).

2. Hugging Face models — downloaded automatically on first use (no action). Used by specific features; transformers/huggingface_hub fetch them the first time you trigger the feature (needs internet + disk, then cached under ~/.cache/huggingface). You do not pre-download these.

Model	Pulled when you…	Size
sentence-transformers/all-MiniLM-L6-v2	use --strategy late / --embed-backend hf	~80 MB
cross-encoder/ms-marco-MiniLM-L-6-v2	run aop rag search/ask (re-ranking, no Cohere key)	~80 MB
sshleifer/tiny-gpt2	run aop finetune train	~10 MB
distilbert-base-uncased	run aop finetune eval --metrics bertscore	~270 MB

Swap the chat/embedding model anytime via .env (e.g. AOP_DEFAULT_CHAT_MODEL=mistral after ollama pull mistral). Cloud models (OpenAI/Anthropic/Gemini/Mistral) need no download — just set the API key in .env and pass --provider.

---

1. Bootstrap the backend (one command)

cd E:\KRONOS
.\scripts\bootstrap.ps1          # creates .venv, installs aop, seeds .env, runs diagnose

POSIX: ./scripts/bootstrap.sh.

This installs the core backend. The heavy/optional features install via extras (do these once if you want them):

# fine-tuning (Phase 5): torch + peft + trl + eval metrics
.\.venv\Scripts\python.exe -m pip install -e ".\backend[finetune]"
# already included by bootstrap: data, rag, memory, orchestrator, api deps.
# optional source connectors: [mongo] [bigquery] [snowflake] ; token-embeddings: [embeddings]

From here, aop means .\.venv\Scripts\aop.exe. (Tip: $aop = ".\.venv\Scripts\aop.exe".)

---

2. Start the services

docker compose up -d            # Qdrant, Redis, Postgres, Redpanda(+Console), Mongo, Langfuse
ollama serve                    # if not already running as a service
.\.venv\Scripts\aop.exe diagnose

diagnose is your green-light dashboard. Everything required should be OK; optional cloud keys show missing (that's fine). Useful UIs once up:

• Redpanda Console — http://localhost:8080
• Langfuse (traces) — http://localhost:3001 (login dev@aop.local / aop-dev-password)

To run the HTTP API (needed for the frontend and REST/WebSocket exploration):

.\.venv\Scripts\aop.exe serve   # http://127.0.0.1:8000  (OpenAPI docs at /docs)

To run the frontend (Canvas + Fine-Tuning UI):

cd frontend
pnpm install                    # first time
pnpm dev                        # http://localhost:3000

---

3. Feature tour (by phase)

Phase 0 — Unified LLM adapter

aop providers                                   # list providers + which have keys
aop models --provider ollama                    # models the daemon offers
aop chat "Explain RAG in one sentence."         # streams from llama3.2:3b
aop chat "Reply 'pong'." --no-stream -t 0       # non-streaming
aop embed "vectorize me"                         # nomic-embed-text → 768-dim vector

Switch providers by setting a key in .env (e.g. AOP_OPENAI_API_KEY=...) then aop chat "..." --provider openai --model gpt-4o-mini.

Phase 1 — Data infrastructure (ingest → search → warehouse → stream)

# Ingest a document (chunk → embed → index into Qdrant + BM25)
aop ingest backend\tests\fixtures\rag.md -c docs

# Ingest tabular data → DuckDB warehouse + searchable
aop ingest backend\tests\fixtures\people.csv -c people --table people

# Other source types (type inferred from extension; --type to force)
aop ingest backend\tests\fixtures\page.html -c web
aop ingest backend\tests\fixtures\talk.srt -c media
aop ingest orders -t sql --query "SELECT * FROM orders LIMIT 100" --table orders
aop ingest db.collection -t mongo --table coll          # needs [mongo]

# Chunking strategies
aop ingest backend\tests\fixtures\rag.md -c late_docs --strategy late --embed-backend hf
aop ingest backend\tests\fixtures\rag.md -c tree_docs --strategy raptor

# Explore
aop collections                                  # vector collections + counts
aop search "how does hybrid retrieval work?" -c docs -k 5
aop warehouse tables
aop warehouse query "SELECT role, COUNT(*) FROM people GROUP BY role"

# Streaming (terminal A then B)
aop stream consume --collection live --idle-timeout 30
aop stream produce "a document that arrives over the stream" --topic aop.documents

Phase 2 — RAG + memory

# Hybrid (dense+sparse, RRF) retrieval + cross-encoder re-rank
aop rag search "combine dense and sparse retrieval" -c docs -k 5
# Grounded, cited answer
aop rag ask "What is re-ranking and why use it?" -c docs
# Per-agent isolation (physical collection agent7__docs)
aop rag ask "..." --namespace agent7 -c docs

# Conversation memory (Postgres turn log + summary + Qdrant episodic recall)
aop memory add user "My project is KRONOS and I prefer Python" -n agent7 -s s1
aop memory chat "Which language do I prefer?" -n agent7 -s s1
aop memory context --query "project" -n agent7 -s s1

Phase 3 — Orchestrator (tools + agents + supervisor)

aop agents                                       # 4 agent types + capabilities
aop tools                                        # MCP tool manifests
# Supervisor routes to the best agent; --steps shows the reasoning trace
aop agent run "Explain re-ranking using the knowledge base, then critique it." --steps
aop agent run "What is a vector database?" --agent react

Agent runs are traced to Langfuse (http://localhost:3001).

Phase 4 — Canvas UI

1. aop serve (backend) + pnpm dev (frontend) → open http://localhost:3000.
2. Drag agents / tools / collections from the left palette onto the canvas; connect them.
3. Type a task into an agent node, click ▶ Run → watch live status + the Inspector stream the trace, tool calls, tokens, and final answer.
4. Toolbar: Export / Import the canvas as JSON.

Phase 5 — Fine-tuning (needs [finetune] extra)

# Build a dataset (records JSONL with {prompt, completion})
aop finetune dataset mydata --jsonl my_records.jsonl --format sft
# Train a tiny LoRA model on CPU (real training; streams progress)
aop finetune train --dataset data\datasets\mydata\train.jsonl --name myagent --max-steps 20
aop finetune jobs
aop finetune registry
aop finetune eval samples.jsonl --metrics rouge,bertscore,ragas
# QLoRA / RLHF will report "requires a CUDA GPU" on a CPU box (by design).

Or use the Fine-Tuning page: in the Canvas, click Fine-Tuning → (or open /finetune) — launch form, live progress, registry/jobs tables.

Phase 6 — Project import & auto-improvement

# Import a codebase (path, .zip, or git URL) into an isolated copy
aop project import .\some\repo
aop project import https://github.com/org/repo.git
# Analyze (ruff + mypy + AST + deps + data-leakage [+ LLM review])
aop project analyze <id>
aop project analyze <id> --no-review            # fast, deterministic only
aop project report <id> --json
# Improve a file (LLM rewrite → diff preview) then apply to the COPY
aop project improve <id> --file path\to\file.py
aop project apply   <id> --file path\to\file.py --yes

Try it on the platform itself: aop project import backend\aop → analyze.

---

4. The HTTP API (with aop serve running)

Open http://127.0.0.1:8000/docs for interactive OpenAPI. Highlights:

Method	Path	Does
GET	/health /agents /tools /collections	status + catalogs
POST	/agent/run	route + run an agent, return result/trace
POST	/rag/search /rag/ask	hybrid retrieval / grounded answer
POST	/ingest	ingest a source
POST	/runs → WS /ws/runs/{id}	start agent run, stream steps live
POST	/finetune/jobs → WS /ws/finetune/{id}	train, stream progress
GET	/finetune/registry	model versions
POST	/projects/import /{id}/analyze /{id}/improve /{id}/apply	project tooling

Example:

curl http://127.0.0.1:8000/agents
curl -X POST http://127.0.0.1:8000/rag/ask -H "content-type: application/json" `
     -d '{\"query\":\"what is re-ranking?\",\"collection\":\"docs\"}'

---

5. Configuration

All settings are environment variables prefixed AOP_, read from .env (copy from .env.example in the source repository). Common ones:

• AOP_DEFAULT_CHAT_MODEL, AOP_DEFAULT_EMBED_MODEL — local model routing.
• AOP_OPENAI_API_KEY / …ANTHROPIC… / …GEMINI… / …MISTRAL… / …COHERE… — opt-in cloud.
• AOP_CHUNK_SIZE, AOP_RAG_TOP_K, AOP_MEMORY_WINDOW_TOKENS — pipeline tuning.
• Service endpoints (AOP_QDRANT_URL, AOP_POSTGRES_DSN, AOP_REDIS_URL, …) — default to the docker-compose stack.

---

6. Quality gates (for contributors)

.\.venv\Scripts\ruff.exe check backend
.\.venv\Scripts\ruff.exe format --check backend
cd backend; ..\.venv\Scripts\mypy.exe aop; cd ..        # strict types
cd backend; ..\.venv\Scripts\python.exe -m pytest -m "not live"; cd ..   # offline
cd backend; ..\.venv\Scripts\python.exe -m pytest -m live; cd ..         # needs the stack
cd frontend; pnpm lint; pnpm typecheck; pnpm build; cd ..

---

7. Troubleshooting

Symptom	Fix
aop diagnose shows a service DOWN	docker compose up -d; for Ollama, ollama serve.
default chat/embed model MISSING	ollama pull llama3.2:3b / ollama pull nomic-embed-text.
frontend palette empty / "backend offline"	start aop serve; check NEXT_PUBLIC_API_BASE.
ModuleNotFoundError: peft/trl (fine-tuning)	pip install -e ".[finetune]".
fine-tuning: requires a CUDA GPU	expected on CPU for QLoRA/RLHF; use sft_lora/dpo.
Langfuse 500 on first boot	give it ~30s to migrate; docker compose up -d langfuse.

---

8. Where to read more

• Architecture decisions: docs/adr (ADR-0001 … 0009 — why each thing is built the way it is).
• Operational runbooks: docs/runbooks — environment, data, RAG+memory, orchestrator, canvas, fine-tuning, project-import.
• Data contracts: docs/schemas — generated JSON Schemas for every model.
• Per-package docs: backend/aop/*/README.md, backend/README.md, and frontend/README.md live with the source in the private repository.
• Project overview + phase status: README.md.