Architecture and trust boundaries

From cognokratos/simple-agent-template · docs/ARCHITECTURE.md · pinned revision c66ce19d7b0c

A template for a secured, observable, evaluated LLM agent. The sample application triages customer-support tickets; everything that is not the sample is meant to be reused unchanged.

Overview

flowchart TB
    B([Browser])

    subgraph HOST [Published on loopback]
        UI["assistant-ui<br/>Next.js :3000"]
        KC["Keycloak<br/>OIDC :8082"]
        ML[("MLflow :5000<br/>traces, experiments,<br/>prompt registry")]
        OC["OpenTelemetry<br/>Collector :4318"]
    end

    subgraph PRIVATE [No published ports]
        GW["Rust gateway / BFF<br/>OIDC+PKCE, session, CSRF,<br/>schema, identity headers"]
        subgraph AG [NAT agent]
            AUTH["service-key + identity<br/>middleware"]
            GR["NeMo Guardrails<br/>input / output rails"]
            RT["ReAct runtime<br/>(bounded loop)"]
        end
        MCP["Rust MCP server<br/>2 read-only tools<br/>(+ optional approval verifier)"]
        DB[("PostgreSQL<br/>authoritative state,<br/>append-only audit")]
        EV["evaluator<br/>(profile: evaluation)"]
    end

    LLM{{"LLM endpoint<br/>OpenAI-compatible<br/>(agent + guard model)"}}

    B -->|"cookie + CSRF"| UI
    B -.->|login redirect| KC
    UI -->|gateway_net| GW
    GW -->|auth_net| KC
    GW -->|"agent_net · Bearer AGENT_API_KEY<br/>+ x-authenticated-*"| AUTH
    EV -->|"agent_net · Bearer AGENT_API_KEY<br/>+ synthetic principal"| AUTH
    AUTH --> GR --> RT
    RT <-->|prompts, tool calls| LLM
    GR <-->|self-check| LLM
    RT -->|"mcp_net · Bearer MCP_API_KEY"| MCP
    MCP -->|"data_net · parameterized SQL"| DB
    AG -.->|"telemetry_net · OTLP"| OC -.-> ML
    EV -.->|results, provenance| ML

    classDef prob fill:#fde68a,stroke:#b45309,color:#000
    class LLM prob

The LLM (yellow) is the only probabilistic component. It is reached only from inside the agent, sees no credentials or identity, and can affect state only through the MCP tools. The sections below give the same picture as text, with the question each boundary answers.

The request path

browser
  │  session cookie + CSRF header, same-origin only
  ▼
assistant-ui (Next.js)            publishes :3000
  │  server-side fetch, no browser headers forwarded
  ▼
Rust gateway (BFF)                publishes nothing
  │  service credential + gateway-minted identity headers
  ▼
NAT (NeMo Agent Toolkit)          publishes nothing
  │  service credential
  ▼
Rust MCP server                   publishes nothing
  │
  ▼
PostgreSQL                        publishes nothing

Alongside it:

NAT  ──OTLP──▶  OpenTelemetry Collector  ──▶  MLflow
evaluator ──▶  NAT (service credential, bypassing the browser path)

What each boundary is for

The two service credentials are not redundant with network isolation. Network membership answers can this packet arrive; it cannot answer is this caller the gateway. NAT trusts the identity headers it receives — they end up in audit records and in signed approval tokens — so it must authenticate its callers.

Network segmentation

Seven Compose networks, each one trust relationship:

The consequences are asserted twice: statically from the resolved configuration (scripts/verify_security_config.py, run by make security-config-test) and at runtime against the live cluster (make network-test).

None of these is internal: true. That flag removes a network's default route — it blocks egress — and does nothing for inbound reachability, which ports: already governs. The agent must reach the model endpoint, so marking its networks internal would break inference while adding no protection this topology does not already have.

Where the model is, and is not, trusted

The model chooses which read-only tools to call and what to say. It does not choose:

• who the user is — identity comes from gateway-minted headers, never from the conversation;
• whether a change is authorized — that requires a signed token the model cannot mint (see APPROVALS.md);
• what a policy decides — backend policy is re-evaluated at the point of mutation, after the human approves;
• what reaches the client — output rails run between the model and the stream.

The agent runtime is trusted, unlike the model. It holds the service credential for the MCP server (MCP_API_KEY) and, when approvals are enabled, the approval signing secret (HITL_APPROVAL_SECRET). Neither enters the model's context or a tool argument. A compromise of the runtime is therefore a different and more serious failure than a manipulated model; see APPROVALS.md — the trust model.

Text that arrives through a tool result is data, never instruction. That is a property the evaluation suite measures rather than asserts: see the injection suite in EVALUATION.md.

Component map

Building a domain application on this

See EXTENDING.md. In short: the support-tickets domain lives in db/init.sql, the MCP tools, agent/config.yml's prompt and tool list, and the evaluation datasets. Everything else is infrastructure.