Picnic — cutting API latency 47% by consolidating...
The Picnic social platform served 1M+ users across a graph of Go microservices behind a GraphQL gateway. The latency win came from a counter-intuitive move: fewer services, tighter contracts.
Archive
337 posts · Page 20 of 29. ← Blog
Runtime Context
Passing per-request values to tools without leaking them into the model's schema.
Pass per-run values via function_invocation_kwargs and read them from FunctionInvocationContext inside tools and middleware, with session.state for state that persists across runs.
step06 · Persisted Conversation
How to serialize an agent.Session to storage and resume the same conversation later, even from a new process.
Serialize an agent.Session with encoding/json, store the bytes, and reload them into a fresh Session in another process to resume the exact same conversation.
Picnic — 80%+ test coverage and Prometheus observability...
Test coverage and observability are the boring infrastructure that makes the interesting changes safe. Notes on how the Picnic team built both, and the on-call experience they enabled.
Shared State
Passing data between middleware by putting them on one object.
Function middleware has no built-in shared bag, so put both middleware on one object and let them read and write instance attributes. That instance is the shared state.
Structured Output: Typed Agent Results
How to make the agent return a typed Go struct instead of free-form prose, via a JSON schema derived from your type.
Two ways to get typed output from an agent: per-run agent.WithStructuredOutput, or agent.WithResponseFormat with jsonformat.MustFor baked into Config.RunOptions.
Globe — running a 30K+ TPS transaction platform on...
The transaction engine had to absorb 30K+ TPS across partner integrations, never lose a transaction, and survive partial failures. The architecture: Go, Kafka, Pub/Sub, Redis, K8s, with idempotency at every layer.
Exception Handling
Catching tool failures in middleware and turning them into graceful replies.
Wrap call_next() in try/except inside FunctionInvocationContext middleware to catch tool exceptions like TimeoutError and set context.result to a friendly fallback.
02 · step04 — Function Tools With Approvals
How to pause a run for human approval before the framework is allowed to execute a tool.
Wrap a tool with tool.ApprovalRequiredFunc so the run pauses and returns a ToolApprovalRequestContent; approve or decline, then resume with RunMessage on the same session.
Idempotency at three layers — the pattern that kept the...
A single layer of idempotency will eventually fail. Three independent layers gives you a margin. Here is the pattern that worked across ingest, worker, and emit boundaries.
Result Overrides
Rewriting a run's output after the model finishes, without touching instructions.
Result-override middleware rewrites context.result after call_next(); chat middleware overrides each ChatResponse and agent middleware wraps the whole AgentResponse.
02 · step03 — Using Function Tools
How to wrap a plain typed Go function as a tool the model can call mid-run.
Wrap a plain typed Go function with functool.MustNew and attach it via agent.Config.Tools so the model can call it; the SDK infers the JSON schema from the signature.