Skip to content

ADR-0003: Local-LLM-first with Ollama; cloud providers opt-in

  • Status: Accepted
  • Date: 2026-06-04
  • Deciders: AOP core

Context

The platform targets a default workflow that runs entirely on a developer's machine — no API keys, no per-token cost, no data leaving the host — while still allowing cloud models when a user explicitly opts in.

Decision

  • Ollama is the default provider for both chat (llama3.2:3b) and embeddings (nomic-embed-text), configured via AOP_DEFAULT_* settings.
  • Cloud providers are opt-in: an adapter with no API key reports UNCONFIGURED from health() and raises ProviderNotConfigured from any networked call — it never silently falls back or mocks a response.
  • All configuration flows through aop.config.Settings (env / .env, AOP_-prefixed). Settings.has_key(provider) is the single source of truth for whether a provider is usable.
  • aop diagnose reports, per provider, whether it is reachable/configured and — for anything not OK — a concrete fix path.

Consequences

  • Positive: works offline out of the box; no secrets required for Phase 0–4 development; privacy by default.
  • Positive: cost and data-egress are explicit, opt-in decisions.
  • Negative: local model quality/latency varies by hardware; the smallest models (e.g. llama3.2:3b) are weaker than frontier cloud models. Users who need higher quality set a cloud key and flip the default provider.
  • Operational: the only hard requirement for Phase 0 is a running Ollama daemon with the two default models pulled.