Production AI Reference Architecture

A composed, guardrailed request path — model + retrieval + tools, observed end to end

Production AI Reference Architecture A composed, guardrailed request path — model + retrieval + tools, observed end to end Observed & governed request path (traces · cost · quality · drift) Client · user / app · Architecture component Client user / app API Gateway · orchestrator · Observed & governed request path (traces · cost · quality · drift) API Gateway orchestrator Guardrails · inline safety I/O · Observed & governed request path (traces · cost · quality · drift) · in + out Guardrails inline safety I/O in + out Foundation Model · served endpoint · Observed & governed request path (traces · cost · quality · drift) Foundation Model served endpoint Retrieval · vector index / KB · Architecture component Retrieval vector index / KB Tools · functions / APIs · Architecture component Tools functions / APIs request retrieve context call tools Legend External Backend Security Cloud Database

Request path

  • • Client → gateway → guardrails → model
  • • Guardrails enforce input and output inline
  • • Not a post-hoc filter

Grounding & action

  • • Retrieval grounds the model in your data
  • • Tools let it act, under contracts
  • • Both inside the governed path

Operated under

  • • Traces, cost, quality, and drift telemetry
  • • Capacity plan + SLOs
  • • Rollback + kill-switch

Architecture diagram • Built with Archify • Create yours ↗ • Hover to trace • R route • Click to focus • +/− zoom • M radar • T theme • E export