Descriptive architecture reference for the coach AI layer (apps/api/app/coach_ai/). This document explains how the system works; the normative rules for changing it live in AI_CONVENTIONS.md, and the chat UI is governed by DESIGN_CONVENTIONS.md.
The AI coach is an LLM-powered assistant for fitness and nutrition guidance. It is built as an orchestrated tool-calling loop around a pluggable LLM provider: every input and output passes a safety policy engine, every tool call is cached, logged, and surfaced to the user as a trace, and chat streams over SSE with a typed event contract.
flowchart TB
UserMsg[User message] --> InputCheck[Policy engine: check_input]
InputCheck -->|block / flag| SafetyMsg[Safety message, no LLM call]
InputCheck -->|pass| Orch[Orchestrator]
Context[ContextBuilder: compact user summary] --> Orch
Orch --> Factory[create_provider]
Factory --> Provider[OpenAIProvider / AnthropicProvider]
Provider -->|tool calls| Registry[Tool registry]
Registry --> DB[(PostgreSQL)]
Registry --> Cache[(Redis)]
Registry -->|results| Provider
Provider -->|final text| OutputCheck[Policy engine: check_output]
OutputCheck --> Response[Response + structured disclaimers]
| Component | File |
|---|---|
| API endpoints | router.py |
| Service (sessions, policies, streaming contract) | service.py |
| Orchestrator (LLM/tool loop) | orchestrator.py |
| User context | context_builder.py |
| Providers | providers/ (factory, model config, OpenAI, Anthropic) |
| Tools | tools/ (registry + 6 tools) |
| Safety policies | policies/ (engine, 4 policies, shared extraction) |
| Prompts and disclaimers | prompts/ |
| Request/response schemas | schemas.py |
| Database models | models.py |
Endpoints (/api/v1/coach, all authenticated): POST /chat, POST /chat/stream, POST /plan, GET /insights. See API.md.
Providers subclass LLMProvider (providers/base.py) and are constructed only by create_provider() in providers/factory.py, selected by settings.llm_provider (openai or anthropic, default openai). Selecting anthropic without ANTHROPIC_API_KEY raises at construction time. Requests use settings.llm_timeout_seconds (default 60). The shared currency between the orchestrator and providers is the internal Message, ToolDefinition, and LLMResponse types; provider-specific shapes (OpenAI tool dicts, Anthropic content blocks, stop-reason mapping) stay inside each provider.
Model IDs and token budgets live in providers/model_config.py, keyed by provider and subscription tier (ModelTier: free, standard, premium). The tier comes from AISession.model_tier (default standard); unknown tiers fall back to standard.
| Tier | OpenAI | Anthropic | max_tokens |
|---|---|---|---|
| free | gpt-4o-mini |
claude-haiku-4-5 (Claude Haiku 4.5) |
1000 |
| standard | gpt-4o |
claude-sonnet-5 (Claude Sonnet 5) |
2000 |
| premium | gpt-4o |
claude-sonnet-5 (Claude Sonnet 5) |
4000 |
OpenAI tiers use temperature 0.7. Anthropic tiers keep temperature unset: Claude Sonnet 5 rejects non-default sampling parameters, so the Anthropic provider omits the field entirely.
The coach has 6 tools, all internal and deterministic (they only read the user’s own data; there is no consent machinery and no external tool category in use). Tools subclass BaseTool and are registered with the ToolRegistry, which owns Redis caching (key: hash of tool name, user ID, and arguments; per-tool TTL) and error handling. Callers never invoke tool objects directly.
| Tool | Purpose | Parameters | Cache TTL |
|---|---|---|---|
get_user_profile |
Demographics, goals, diet preferences | none | 5 min |
get_recent_checkins |
Daily check-ins (weight, energy, sleep, mood) | days (1-90, default 14) |
1 min |
get_weight_trend |
7-day moving average, weekly rate of change | days (7-365, default 30) |
5 min |
get_nutrition_summary |
Average calories and macros, logging rate | days (1-90, default 14) |
2 min |
calculate_tdee |
BMR, TDEE, and macro targets | optional weight_kg |
10 min |
get_adherence_metrics |
Check-in and logging rates, streaks | days (7-90, default 14) |
5 min |
Every execution, cached or not, is logged to ai_tool_call_log and surfaced to the user as a tool trace (name, description, input/output summaries, latency, cache status).
SafetyPolicyEngine (policies/engine.py) runs four policies in severity order on both user input (check_input, before the model sees the message) and model output (check_output):
calorie_minimum disclaimer.weight_loss_rate disclaimer.medical disclaimer attached.Policy actions: ALLOW (pass), MODIFY (pass with disclaimer or rewritten content), BLOCK (replace the message), FLAG (replace with support resources). When an input check returns BLOCK or FLAG, the service returns the safety message without calling the LLM. On the streaming path the output check runs after streaming completes: already-streamed text is not retracted, but violations are still logged and disclaimers still attach to the done event.
Shared parsing helpers live in policies/extraction.py (extract_calorie_values, extract_weekly_weight_change_kg, unit-aware including lb-to-kg conversion); policies never re-implement number or unit parsing.
Disclaimers are structured data. Policies attach disclaimer texts from prompts/disclaimer_templates.py (keys: general, medical, calorie_minimum, weight_loss_rate, fetched via get_disclaimer()). They surface as ChatResponse.disclaimers and the done event’s disclaimers field; they are never appended into the message body.
Every failed check writes an ai_policy_violation_log row.
Prompt text lives in prompts/system_prompts.py: COACH_SYSTEM_PROMPT (personality, tool-use guidance, safety rules mirroring the policies) and PLAN_SYSTEM_PROMPT (weekly plan generation), selected with get_system_prompt(). An insights prompt exists but the /coach/insights endpoint currently computes insights in Python from the user context, without an LLM call.
ContextBuilder (context_builder.py) loads profile, goals, check-ins, nutrition, and adherence data and renders a compact single-line summary that is appended to the system prompt (roughly 40% fewer tokens than a verbose context).
Sessions are ai_session rows created and resumed only through CoachService._get_or_create_session:
settings.coach_session_idle_hours (default 24) marks it EXPIRED and creates a fresh session._record_exchange appends each user/assistant pair and trims stored history to the last settings.coach_max_conversation_history messages (default 12).PROMPT_HISTORY_MESSAGES = 6 stored messages (3 exchanges).message_count and tokens_used; datetimes are naive UTC throughout.POST /coach/chat)MAX_TOOL_ROUNDS = 5 rounds: call the provider with all tool definitions; if the response requests tools, execute them through the registry, append the results, and loop; otherwise the round’s text is the final answer. Hitting the cap returns an apology with finish_reason="max_iterations".ChatResponse with message, session_id, tool_trace, confidence, data_gaps, disclaimers, tokens_used.POST /coach/chat/stream)SSE events are JSON objects {type, data} with type exactly one of token | tool_start | tool_end | done | error. The stream runs the same multi-round tool loop: each round streams token events, then emits tool_start/tool_end around each tool execution, and continues until a round produces no tool calls.
The service (not the router) emits the terminal done event:
{
"session_id": "uuid",
"confidence": 0.85,
"data_gaps": [{"field": "...", "description": "...", "suggestion": "..."}],
"disclaimers": ["..."],
"tool_trace": [{"tool_name": "...", "latency_ms": 45, "cached": false}]
}
On failure the client receives a single error event with a generic user-safe string (never str(e)), and the partial exchange is not persisted. The mobile SSE client (apps/mobile/src/services/api/coachService.ts, useCoach.ts) depends on every detail of this contract; see AI_CONVENTIONS.md rule 5 before changing it.
confidence is a data-completeness score (mean of check-in coverage, nutrition coverage, and profile completeness over the last 7 days), not a model self-assessment. data_gaps lists what would improve advice: fewer than 7 check-ins or nutrition days, missing height, or missing activity level. The insights endpoint uses a stricter 14-day variant as its data_quality_score.
Two tables make the coach observable (schema: DATABASE.md):
ai_tool_call_log: one row per tool execution, written by the orchestrator: tool name and category, input hash and summary, output summary, status, error message, latency, cache hit.ai_policy_violation_log: one row per failed policy check (input or output): violation type, severity, sanitized trigger content, action taken.Useful queries:
-- Most used tools, last 7 days
SELECT tool_name, COUNT(*) AS calls, AVG(latency_ms) AS avg_latency
FROM ai_tool_call_log
WHERE created_at > NOW() - INTERVAL '7 days'
GROUP BY tool_name ORDER BY calls DESC;
-- Policy violations by type, last 30 days
SELECT violation_type, severity, COUNT(*) AS count
FROM ai_policy_violation_log
WHERE created_at > NOW() - INTERVAL '30 days'
GROUP BY violation_type, severity ORDER BY count DESC;
New tools, policies, providers, or streaming changes must follow AI_CONVENTIONS.md: it defines the provider factory boundary, where model names may live, tool registration and logging requirements, policy coverage and severity ordering, the streaming contract, session caps, and the tests each kind of change must ship with.