Every AI financial system is built on data, and in finance the data is uniquely treacherous: expensive, fragmented, revised after the fact, survivor-biased, and riddled with subtle ways to accidentally see the future. Getting the data foundation right matters more than any model — a brilliant model on leaky or biased data is a confident path to losing money. This post is about the financial data landscape and the traps that make it the hardest, most decisive part of the work.
Every financial AI system lives or dies on its data, and financial data is uniquely treacherous: expensive, fragmented, revised, survivor-biased, and riddled with ways to accidentally see the future. The landscape (market, document, proprietary data) and the traps — lookahead, survivorship, point-in-time correctness — that make it the most decisive part. (Educational, not financial advice.)
Attention is the operation that makes transformers powerful, and for years it was also their worst performance bottleneck — not because it does too much math, but because it moved too much data. FlashAttention is the single most important GPU optimization of the LLM era, and it's a perfect case study in everything this series has built: it's memory-bound, the roofline says so, and the fix is to raise arithmetic intensity by never writing the big intermediate to HBM. Understanding it is understanding how real kernel optimization works.
Naive attention materializes an N×N matrix in HBM — memory-bound and quadratic. FlashAttention fixes it with pure roofline reasoning: tile into shared memory, use an online softmax, fuse the kernel, never touch HBM. The archetype of all inference optimization.
Financial engineering has always been computational, but a new layer is reshaping it: artificial intelligence applied not just to predict prices but to read documents, structure judgment, monitor risk, and preserve institutional decision-making. AI financial engineering is the discipline of building these systems end to end — and doing it in an industry where being wrong costs real money and being unexplainable is often illegal. This series builds that discipline from first principles to production.
Financial engineering has always been computational; AI now extends it into reading documents, structuring judgment, and monitoring risk. What AI financial engineering is — the synthesis of model-driven quant and data-driven AI — and why finance (real money, mandatory explainability, auditability, adversarial data, human judgment) makes it uniquely hard. (Educational, not financial advice.)
The fastest way to speed up LLM inference is often not a better algorithm but fewer bits per number. Precision — how many bits represent each weight and activation — is a direct lever on both of the limits this series has built up: it halves or quarters the data you move (bandwidth, post 2) and unlocks the GPU's fastest arithmetic units (compute, post 3). Understanding floating-point formats is understanding why fp16, bf16, and int8 are everywhere in modern inference.
FP32, FP16, BF16, and INT8/INT4 — how a float splits bits between range and precision, why bf16 won, and why fewer bits is a double win: less data to stream from HBM and faster tensor-core math. Quantization and why LLMs tolerate it.
This closing post makes everything concrete: the anatomy of a skill definition, the templates that get you started fast, and the synthesis of the series into a practical approach you can apply today. It turns the principles — scope, context discipline, specificity, reliability, operation — into a reference you can build from.
The concrete reference: the anatomy of a skill definition (the SKILL.md pattern — metadata/trigger, instructions, knowledge, resources), starter templates (general, domain, workflow, integration), and the series' synthesis — make agents reliably useful by building and tending a library of captured expertise.
To make a GPU fast you have to express your problem the way the hardware wants to run it: as thousands of near-identical threads marching in lockstep. CUDA is the programming model that exposes this, and its abstractions — threads, warps, blocks, grids — map directly onto the SMs and cores from post 1. You don't need to write kernels to be an inference engineer, but you do need this model to understand why some operations fly and others crawl.
Threads, warps, blocks, and grids; SIMT and branch divergence; and the cleverest trick — hiding slow memory by oversubscribing warps (occupancy). Why a GPU can be underutilized without being either compute- or memory-bound.
The gap between a skill that works in a demo and one that works reliably in daily use is where most of the real lessons live. Skills in the wild face messy inputs, edge cases, drift, and the challenge of being maintained and shared across a team over time. This post is about what actually works when skills meet reality — the patterns that hold up, the failure modes that recur, and the operational discipline that keeps a skill library useful.
The gap between a demo skill and one that works in daily use is where the real lessons live. What makes skills succeed in the wild (the fundamentals), the failure modes that recur (drift, overlap, trigger misfires, bloat, over-trust), and operating the library as a maintained, curated, shared system.
Every AI forward deployed engineer builds one-offs — a bespoke deployment for one customer's data, workflow, and trust. The ones who create lasting value turn those one-offs into product: the patterns that repeat become a platform, the platform makes the next deployment faster, and the field learnings flow back to shape what gets built. This closing post is about the flywheel that turns bespoke AI work into a compounding asset, and the career arc of the engineer who runs it.
Every AI FDE builds one-offs — a bespoke deployment for one customer's data, workflow, and trust. The ones who create lasting value turn those one-offs into product: the patterns that repeat become a platform, the platform makes the next deployment faster, and field learnings flow back to shape what gets built. The flywheel that turns bespoke AI work into a compounding asset — and the AI FDE career arc.
Posts 1 and 2 kept drawing a line between two worlds: workloads limited by how fast the GPU can compute, and workloads limited by how fast it can move data. The roofline model is the single tool that tells you which world you're in for any given operation — and therefore which optimizations will help and which are a waste of time. It is the most useful mental model in performance engineering, and it fits on one chart.
The roofline model answers the one question that governs every optimization — compute-bound or memory-bound? — from arithmetic intensity (FLOPs per byte). Why LLM prefill is compute-bound and decode is memory-bound, and why batching is the master throughput lever.
Where general business skills capture how you do ordinary work, domain-specific skills capture deep expertise in a particular field — the specialized knowledge, rules, and judgment of law, medicine, finance, a specific codebase, a regulated process. These are the highest-value and highest-stakes skills: they let an agent operate in a specialized domain, but they demand the most rigor, because being wrong in a specialized domain is both more likely and more costly. This post is about capturing deep domain expertise as skills.
Domain-specific skills capture deep field expertise — law, finance, a specific codebase, a regulated process. The highest-value and highest-stakes skills, demanding the most rigor: grounded authoritative knowledge (not the model's hallucinations), precisely-stated rules, verification, and human oversight.
A pilot that works is not a system the customer can run. Productionizing AI means making it reliable, affordable, fast, and observable enough to be real infrastructure — and then handing it over so the customer operates it without you. The forward deployed engineer's goal, in the end, is to make themselves unnecessary: a deployment that only works while you're standing next to it hasn't actually been delivered.
A pilot that works is not a system the customer can run. Productionizing AI means making it reliable, affordable, fast, and observable enough to be real infrastructure — then handing it over so the customer operates it without you. The FDE's goal, in the end, is to make themselves unnecessary. Cost, latency, drift, observability, and a handover that transfers the eval discipline, not just the code.
Post 1 ended on a reversal: GPUs are fast at arithmetic, but arithmetic is rarely the bottleneck — moving data is. This post makes that concrete by walking the GPU's memory hierarchy, from the tiny ultra-fast registers next to the compute units down to the big slow memory that holds the model's weights. Once you see the orders-of-magnitude gaps between these levels, every performance decision in LLM inference starts to make sense.
The GPU memory hierarchy from registers to HBM, the difference between capacity and bandwidth limits, and the defining fact of LLM generation: it's memory-bandwidth-bound because every token reads the whole model from HBM — which is why batching and quantization work.