1046 posts · Page 72 of 88. ← Blog
How to make the model return a typed Go struct instead of prose — hand RunText a pointer, get back populated fields.
Pass RunText a pointer to a Go struct via WithStructuredOutput and the framework asks the model for matching JSON, then unmarshals it straight into your fields.
The philosophy, trade-offs, and what we learned converting 18+ agents in 3 months.
The philosophy, trade-offs, and what we learned converting 18+ agents in 3 months. Provider abstraction as the foundation for portable agents.
Caching giveth performance and taketh away your sanity. The same technique that makes systems fast introduces a whole category of subtle, intermittent, hard-to-debug problems — stale data that appears randomly, a cache that collapses under load at the worst moment, bugs that only happen when the cache is cold or full. This closing post catalogs the pitfalls that bite real systems, and distills the practical wisdom of the series: cache deliberately, expect the failure modes, and remember that the two genuinely hard things are still hard.
Caching giveth performance and taketh away your sanity. The same technique that makes systems fast introduces subtle, intermittent, hard-to-debug problems — stale data appearing randomly, a cache collapsing under load, bugs that only happen when the cache is cold. This catalogs the pitfalls and the practical wisdom.
How to gate a sensitive tool behind human consent: the run pauses, hands you an approval request, and only calls the tool once you approve.
Wrap a tool with ApprovalRequiredFunc and the run pauses with a ToolApprovalRequestContent; approve it, feed the decision back on the same session, and the tool fires.
Every time a web page loads instantly on a repeat visit, or a video streams smoothly from halfway around the world, caching is the reason. The web is layered with caches — in your browser, at CDN edge servers near you, in front of origin servers — all applying the same caching principle to make the internet fast. And remarkably, much of it is coordinated by a few HTTP headers that let servers tell caches exactly what to store and for how long. Understanding web and CDN caching is understanding how the internet stays fast at global scale.
Every time a web page loads instantly on a repeat visit, or a video streams smoothly from across the world, caching is the reason. The web is layered with caches — browser, CDN edge, origin — all applying the same principle, much of it coordinated by a few HTTP headers.
How to wrap an ordinary Go function as a tool the model can call, with the schema derived automatically from its types.
Wrap a plain Go func with functool.MustNew and the framework derives its JSON schema and drives the tool call inside a single RunText — two model round-trips, one call.
A cache inside a single application process is easy — but it doesn't scale, and every instance of your app has its own separate copy. The moment you run many application servers, you want a shared cache they all use, which means a cache that lives across the network on its own machines: a distributed cache like Redis or Memcached. This unlocks scale and sharing, but introduces the distributed-systems problems that a local cache never had. Understanding distributed caching is understanding how caching works at real scale.
A cache inside a single process is easy — but it doesn't scale, and every instance of your app has its own separate copy. Run many servers and you want a shared cache across the network on its own machines: a distributed cache like Redis or Memcached. This unlocks scale and sharing, but introduces distributed-systems problems.
How to keep a conversation's history on the Foundry service instead of in local memory, so the client never resends the transcript.
Create a Foundry project conversation, bind its ID into a session with WithServiceID, and let the service keep the transcript across turns without resending it.
How to instrument multi-agent systems with OpenTelemetry, propagate trace context across an in-memory bus, and build a layered evaluation pipeline — from real-time policy gates to async LLM-as-judge to SLO-based trust scoring. Everything I learned building Genie.
Knowing to cache is one thing; wiring the cache into your application correctly is another. Should the application manage the cache itself, or should the cache sit transparently in front of the source? Should writes go to the cache, the database, or both — and in what order? These questions have standard answers — the caching patterns — and choosing the right one shapes your consistency, performance, and complexity. Getting the pattern right is how caching goes from "store some stuff" to a coherent, correct design.
Knowing to cache is one thing; wiring the cache into your application correctly is another. Should the application manage the cache itself, or should it sit transparently in front of the source? Should writes go to the cache, the database, or both? These questions have standard answers — the caching patterns.
How an agent.Session carries conversation history so a second prompt can refer back to the first.
A client-side agent.Session carries conversation history so a second prompt can refer back to the first. Create it with CreateSession, thread it with WithSession.
This is the hard one. "There are only two hard things in computer science: cache invalidation and naming things" names it directly — cache invalidation is genuinely, notoriously difficult. The moment you cache a copy of data, you've created a second source of truth that can drift from the first, and keeping them in sync (or deciding how much drift you'll tolerate) is a problem with no clean, universal solution. Understanding why it's hard, and the strategies for managing it, is the difference between caching that helps and caching that causes baffling bugs.
This is the hard one. 'There are only two hard things in computer science: cache invalidation and naming things' names it directly. The moment you cache a copy of data, you've created a second source of truth that can drift from the first, and keeping them in sync has no clean universal solution.