Archive
1290 posts · Page 14 of 108. ← Blog
Most of your software is dependencies, so most of your risk is too. Dependency security is the practical, everyday discipline of controlling what you pull in, knowing when it's vulnerable, and updating without breaking — the highest-leverage supply chain work most teams can do. This post covers pinning, scanning, and the update discipline that keeps the dependency tree from becoming a liability.
Most of your software is dependencies, so most of your risk is too. Dependency security is the practical, everyday discipline of controlling what you pull in, knowing when it's vulnerable, and updating without breaking — the highest-leverage supply chain work most teams can do. Pinning, scanning, and the update discipline that keeps the dependency tree from becoming a liability.
Reinforcement learning was a tidy theory stuck on toy problems until it was combined with deep learning. Replacing the value table with a neural network let RL handle the enormous, continuous state spaces of real problems — and the result, deep RL, is what beat humans at Atari, Go, and StarCraft, and what trains agents in the real world today. This post is about that fusion: why it's powerful, why it's notoriously unstable, and the tricks that make it work.
Replacing the value table with a neural network let RL handle huge, continuous, pixel-level state spaces — the leap that beat humans at Atari and Go. Why naive deep RL diverges, the tricks that tame it (experience replay, target networks), and why it's powerful, brittle, and sample-hungry.
To defend the supply chain you have to know how attackers get in — and there are more doors than most teams realize. Malicious packages, hijacked dependencies, dependency confusion, compromised build systems, and poisoned maintainer accounts each exploit a different link in the chain. This post is a field guide to the attack patterns, because each one maps to a specific defense.
To defend the supply chain you have to know how attackers get in — and there are more doors than most teams realize. Malicious packages, dependency confusion, compromised build systems, and poisoned maintainer accounts each exploit a different link. This post is a field guide to the attack patterns, because each one maps to a specific defense.
The elegant idea of value functions becomes a practical algorithm through one trick: learn by comparing your prediction to a slightly-better prediction, and nudge toward the difference. Temporal-difference learning and its famous instance, Q-learning, are how an agent learns the value of its actions purely from experience — no model of the world required. This is the algorithm that first made reinforcement learning work, and its core idea quietly underlies much of the field.
Learn a guess from a better guess: temporal-difference learning and Q-learning nudge value estimates toward reward + γ × value-of-next, model-free, bootstrapping from every step. Off-policy learning, ε-greedy exploration, and why the Q-table can't scale — motivating deep RL.
You don't just ship the code you wrote — you ship the code your dependencies wrote, and their dependencies, and the build system that assembled it all. That entire chain is your attack surface, and attackers have noticed. Software supply chain attacks target the trusted process by which software is built and delivered, and they've become one of the most damaging classes of breach precisely because they weaponize trust. This series is about defending that chain.
You don't just ship the code you wrote — you ship your dependencies' code, and the build system that assembled it all. That entire chain is your attack surface, and attackers have noticed. This opening post explains why supply chain attacks (SolarWinds, Log4Shell, registry attacks) became so damaging, and frames the defense: shift from implicit trust to explicit verification.
The key idea that makes reinforcement learning work is deceptively simple: instead of chasing immediate reward, estimate the long-term value of being in a situation, and act to move toward high-value situations. Value functions are how an agent "thinks ahead," and the Bellman equation is the elegant recursive relationship that makes computing them possible. This is the conceptual core of RL — grasp it and the algorithms become obvious.
The idea that makes RL work: estimate the long-term value of a situation and move toward high-value ones. State-value and Q-functions, the Bellman equation (value = reward + γ × value-of-next) that turns infinite look-ahead into a recursion and solves credit assignment, and planning vs learning.
Git lets you rewrite history — amend a commit, squash a messy branch, drop a secret that was committed by mistake. But "rewrite" is a misnomer: you never edit a commit, you replace it with a new one and move a pointer. Once you understand that, the rules for doing it safely become obvious, and the one genuine danger — rewriting history other people are building on — comes into sharp focus. This closing post turns the whole series into a working discipline.
"Rewrite" is a misnomer: you never edit a commit, you replace it with a new one and move a pointer. Once you see that, the rules for doing it safely become obvious — and the one real danger, rewriting history others build on, comes into sharp focus. The capstone that turns the series into a discipline.
Behind every reinforcement learning algorithm is one formal structure: the Markov decision process. It's the precise way of writing down "an agent making a sequence of decisions under uncertainty," and once a problem is framed as an MDP, the whole toolkit of RL applies to it. This post builds the MDP from its parts and explains the one assumption — the Markov property — that makes the entire theory work.
Behind every RL algorithm is one structure: the Markov decision process (states, actions, transitions, reward, discount). The Markov property that makes it tractable, why state design matters, and reward design as RL's subtle, decisive, error-prone part (reward hacking) — the problem RLHF later solves.
Collaboration in Git is the same object model, stretched across two repositories. A remote is just a named URL; a remote-tracking branch is just a local pointer that remembers where the other side was; fetch and push are just object transfers plus ref updates, governed by a small syntax called the refspec. Nothing new is invented for the network — it's the graph and the pointers, reaching across a wire.
Collaboration in Git is the same object model stretched across two repositories. A remote is a named URL; a remote-tracking branch is a pointer remembering where the other side was; fetch and push are object transfers plus ref updates governed by refspecs. The network adds almost no new concepts.
Supervised learning needs a teacher who knows every right answer; reinforcement learning needs only a goal and the freedom to try. It's how a system learns to play a game, control a robot, or — increasingly — align a language model, not by being shown correct outputs but by acting, seeing what happens, and learning from the consequences. This series builds RL from first principles, and it starts with why "learning from reward" is a fundamentally different and more general kind of learning.
Supervised learning needs a teacher who knows every answer; RL needs only a goal and the freedom to try. Learning from consequences via the agent-environment-reward loop, the policy, delayed reward and credit assignment, and the exploration-exploitation dilemma that makes RL its own discipline.
"I lost my commits" is almost never true. A bad reset, a botched rebase, a deleted branch — the commits are usually still there, sitting unreachable in the object store, waiting out their grace period before garbage collection. The reflog is the log of everywhere your refs have been, and it's the tool that turns "I destroyed my work" into a thirty-second recovery. Knowing it exists changes how fearlessly you can use Git.
"I lost my commits" is almost never true. A bad reset or botched rebase leaves the commits sitting unreachable in the object store, waiting out a grace period. The reflog is the log of everywhere your refs have been — the tool that turns "I destroyed my work" into a thirty-second recovery.
Without measurement, GTM is guessing — you can't tell a channel that works from one that flatters you, a healthy business from one quietly bleeding, or whether your last change helped. But GTM measurement has a trap engineers fall into from the opposite side: drowning in dashboards of vanity metrics that feel rigorous while missing the two or three numbers that actually decide whether the business works. This closing post is about measuring what matters — and the unit economics that separate a real business from an expensive way to lose money.
Without measurement, GTM is guessing — but the engineer's trap is drowning in dashboards of vanity metrics that feel rigorous while missing the two or three numbers that actually decide whether the business works. This is about measuring what matters — and the unit economics that separate a real business from an expensive way to lose money.