Archive
1290 posts · Page 13 of 108. ← Blog
Before a computer-use agent can click anything, it has to understand what's on the screen — and the screen is just an image of pixels designed for human eyes. Turning that image into "there's a submit button here, a text field there, an error message at the top" is the perception problem, and it's where computer-use agents most often succeed or fail. This post is about how agents see a GUI, the two main approaches, and why perception is the foundation everything else rests on.
Before clicking anything, an agent must understand the screen — pixels designed for human eyes. The perception problem, the two approaches (vision-based multimodal interpretation vs structured accessibility/DOM data), why hybrid wins, and why perception is the foundation most failures trace back to.
You can't defend against what you can't categorize. Prompt-based attacks come in a few structurally distinct shapes — direct injection, indirect injection through content the model reads, and jailbreaks that target the model's safety training — and each demands a different defense. This post maps the attack surface so the rest of the series can defend it systematically.
You can't defend what you can't categorize. Prompt-based attacks come in structurally distinct shapes — direct injection, indirect injection through content the model reads, and jailbreaks targeting the model's safety training — and each demands a different defense. This post maps the attack surface so the rest of the series can defend it systematically.
Most AI agents act through APIs and tools built for them. Computer-use agents do something more radical: they operate software the way a human does — looking at the screen, moving the cursor, clicking buttons, typing into fields. This lets an agent use any application, even ones with no API, which is powerful and uniquely hard. This series builds computer-use (GUI) agents from first principles, starting with what they are and why operating the screen is a different, harder kind of agency.
Most agents act through APIs; computer-use agents operate the screen like a human — looking, clicking, typing — which lets them use any software, even API-less apps. Why that universality is powerful, why the human interface is uniquely hard for machines, and why you should prefer an API when one exists.
Prompt injection is the defining security problem of LLM applications, and — unlike SQL injection, which it superficially resembles — it has no clean fix. The reason is structural: a language model reads instructions and data through the same channel, and cannot reliably tell which is which. This opening post explains why that makes injection fundamentally hard, and reframes the goal from "prevent it" to "contain the blast radius."
Prompt injection is the defining security problem of LLM applications, and unlike SQL injection it has no clean fix — because a model reads instructions and data through the same channel and can't reliably tell them apart. This opening post explains why that makes injection fundamentally hard, and reframes the goal from prevent to contain the blast radius.
Individual controls — SBOMs, signing, provenance, dependency scanning — only add up to security when they're assembled into a coherent program with priorities, ownership, and a sensible starting point. This closing post turns the pieces into a practical roadmap: what to do first, how the controls reinforce each other, and how to build supply chain security incrementally without trying to boil the ocean.
Individual controls — SBOMs, signing, provenance, scanning — only add up to security when assembled into a coherent program with priorities, ownership, and a sensible starting point. This closing post turns the pieces into a practical roadmap: what to do first (dependency hygiene and SBOMs), how the controls reinforce each other, and how to build supply chain security incrementally without boiling the ocean.
The build system is where source becomes artifact — and where the most dangerous supply chain attacks strike, because tampering there produces a malicious release that looks completely legitimate. Securing the build means making it isolated, reproducible, and trustworthy enough that its provenance actually means something. This post covers hermetic builds, build isolation, and the practices that make "how it was built" a guarantee rather than a hope.
The build system is where source becomes artifact — and where the most dangerous supply chain attacks strike, because tampering there produces a malicious release that looks completely legitimate. Securing the build means making it isolated, reproducible, and trustworthy enough that its provenance actually means something. Hermetic builds, isolation, and platform-generated provenance.
Signing is how you turn "trust me, this artifact is authentic" into "verify it yourself." A cryptographic signature binds an artifact to its producer and proves it hasn't been tampered with since. But traditional signing has a painful key-management problem, and the modern answer — keyless signing with Sigstore — is what finally made artifact signing practical enough to be routine.
Signing turns 'trust me, this artifact is authentic' into 'verify it yourself.' A signature binds an artifact to its producer and proves it wasn't tampered with. But traditional signing has a painful key-management problem, and the modern answer — keyless signing with Sigstore — is what finally made artifact signing practical enough to be routine. Plus the verification half that makes signatures mean something.
Reinforcement learning is the most seductive paradigm in machine learning: define a goal, let an agent figure out the rest. It's also the hardest to get working and the easiest to misapply. This closing post is the practical judgment the series was building toward — where RL genuinely shines, where a simpler method is the honest choice, and how to recognize which situation you're in. It's the difference between RL as a tool and RL as a trap.
RL is the most seductive ML paradigm and the easiest to misapply. Where it genuinely shines (games, control, alignment), where a simpler method is the honest choice, and how to tell which — the difference between RL as a tool and RL as a trap. Its deepest value: optimizing goals you can judge but not specify.
An SBOM tells you what's in an artifact. Provenance tells you where it came from and how it was built — a tamper-evident record of the source, the builder, and the steps that produced it. It's the direct answer to the SolarWinds problem: a signed release means nothing if the build itself was compromised, so you need verifiable evidence of the build process, not just the output.
An SBOM tells you what's in an artifact; provenance tells you where it came from and how it was built — a tamper-evident record of the source, builder, and steps. It's the direct answer to the SolarWinds problem: a signed release means nothing if the build itself was compromised. This post covers provenance, attestation, in-toto, and the SLSA framework.
The single most consequential application of reinforcement learning today isn't a game or a robot — it's the technique that turned raw, unruly language models into helpful assistants. Reinforcement learning from human feedback solved the problem that you can't write a reward function for "be helpful, honest, and harmless," so you learn one from human judgments instead. This post connects everything in the series to the method that made modern AI assistants possible.
The most consequential use of RL today: you can't write a reward for 'be helpful, honest, harmless', so RLHF learns one from human preference comparisons and optimizes with PPO. The three-stage pipeline, the KL constraint against reward hacking, and the series' capstone — plus its limits (sycophancy, bias).
When the next Log4Shell-scale vulnerability is announced, the first question every organization must answer is "are we affected?" — and the teams that can answer it in minutes instead of weeks are the ones with an SBOM. A Software Bill of Materials is a complete, machine-readable inventory of everything in your software. It's the foundation of supply chain response, and increasingly, a requirement.
When the next Log4Shell-scale vulnerability is announced, the first question every organization must answer is 'are we affected?' — and the teams that can answer it in minutes instead of weeks are the ones with an SBOM. A Software Bill of Materials is a complete, machine-readable inventory of everything in your software: the foundation of supply chain response, and increasingly a requirement.
Value-based methods like Q-learning learn how good each action is, then act greedily. There's a different and often better approach: skip the value estimates and learn the policy itself, directly adjusting the probability of actions to get more reward. Policy gradient methods are how RL handles continuous control and stochastic strategies — and the actor-critic architecture that combines them with value learning is the foundation of the algorithms used to fine-tune today's language models.
Instead of learning values then acting greedily, learn the policy directly: adjust action probabilities to earn more reward. Why this handles continuous and stochastic policies, the high-variance problem, and actor-critic (and PPO) — the algorithm family behind RLHF and LLM alignment.