The Kernel/User-Space Boundary
To understand why eBPF is such a big deal, you have to understand the wall it lets you cross: the boundary between user space and the kernel. That boundary exists for good reasons — safety and isolation — but it also meant that extending the kernel was historically a choice between two bad options. eBPF is the third option nobody had before.
The previous post said eBPF gives you kernel-level power safely. This post explains the boundary that made that power hard to reach, and the old dilemma — modify the kernel, or work around it from user space — that eBPF resolves. Understanding why the kernel is walled off is what makes eBPF’s achievement clear: it’s not just a tool, it’s a new way across a fundamental divide.
Two worlds: user space and kernel space
An operating system splits execution into two privilege levels: - Kernel space — the privileged core. It manages hardware, memory, processes, the network stack, and the filesystem. It sees and controls everything, and its code runs with full hardware privileges. - User space — where your applications run, unprivileged and isolated. An app can’t directly touch hardware or other processes’ memory; it asks the kernel to do privileged things on its behalf via system calls (syscalls).
This separation is deliberate and foundational to system stability and security: a buggy or malicious application can’t take down the machine or read another process’s memory, because it’s confined to user space and mediated by the kernel. The boundary is enforced by the hardware and the kernel, and crossing it (a syscall) is a controlled, checked transition.
The consequence for anyone wanting custom kernel-level behavior: your code, running in user space, is on the wrong side of the wall. You can ask the kernel to do things, but you can’t easily add to what the kernel itself does — and the kernel is exactly where the most powerful visibility and control live.
The old dilemma: modules vs. user space
Before eBPF, getting custom logic to run with kernel-level power meant one of two unappealing options:
Option 1: Kernel modules. Write code that loads into the kernel (a loadable kernel module, LKM). This gives full kernel power — but at serious cost: - A bug crashes everything. Module code runs with kernel privileges and no safety net; a null-pointer dereference or bad memory access panics the entire machine, not just one process. There’s no isolation. - Version fragility. Modules are tightly coupled to specific kernel versions and internals; they break across kernel updates and require careful maintenance. - High barrier and risk. Writing correct kernel code is hard and dangerous, so it’s the domain of specialists, and shipping it is slow and cautious. You’re modifying the most critical software on the machine.
Option 2: Stay in user space. Do everything from an unprivileged application, using the limited interfaces the kernel exposes. This is safe (a crash only takes down your app) but weak: - Limited visibility. You only see what the kernel chooses to expose through existing interfaces, not the rich internal events you often want. - Overhead. Getting kernel data to user space means copying it across the boundary and context-switching, which is expensive at high volume (e.g. processing every packet or every syscall in user space is slow). - Can’t act in the fast path. You can observe after the fact, but you can’t cheaply intervene inside kernel operations.
So the dilemma was stark: kernel power with kernel danger (modules), or user-space safety with user-space limitations. For decades, that trade-off shaped what kinds of tools were feasible — powerful kernel-level tooling was rare, risky, and expert-only, while safe user-space tooling was limited and slow. There was no way to have both power and safety.
eBPF: the third option
eBPF resolves the dilemma by being the third option that was missing: kernel-level power with user-space-like safety. It lets you run your program in the kernel (so you get the visibility, speed, and fast-path control of a module) but safely (so a bug can’t crash the machine, because the verifier proved it can’t).
Concretely, eBPF gives you the best of both sides: - Kernel-side power — your program runs in the kernel, at kernel events, with kernel visibility and speed. Like a module, you’re on the powerful side of the wall. - User-space-like safety — the verifier (post 4) guarantees the program can’t crash the kernel, access invalid memory, or hang. Like a user-space app, a bug is contained (the program is rejected or safely bounded), not catastrophic. - Runtime and portable — loaded and attached at runtime, no reboot; and with modern tooling (CO-RE, post 8), portable across kernel versions rather than version-locked like modules.
This is why eBPF is often described as making the kernel programmable: it turns “the kernel does what its built-in code does” into “you can safely extend what the kernel does, on the fly.” The wall between user and kernel space is still there and still enforced — eBPF doesn’t tear it down — but it provides a safe, controlled gate through which your verified programs can run on the powerful side.
Why this reframing matters
Seeing eBPF against this history is what makes its impact obvious. The wave of eBPF-powered tools (observability, networking, security — posts 5–7) exists because the old dilemma was resolved. Tools that would have required a risky, version-fragile, expert-only kernel module can now be built as safe, portable eBPF programs — so far more people can build far more powerful kernel-level tooling than ever before. That democratization of safe kernel programming is the real story: eBPF didn’t just add a feature; it removed a decades-old trade-off that had constrained what infrastructure tools could do. The rest of the series builds on this — starting with how an eBPF program actually gets from your source into the kernel and runs.
Key takeaways
- The OS splits execution into kernel space (privileged, sees/controls everything — hardware, memory, processes, network) and user space (unprivileged, isolated apps that request privileged actions via syscalls) — a deliberate boundary enforced for stability and security.
- Custom kernel-level behavior historically meant a dilemma: kernel modules (full power, but a bug panics the whole machine, version-fragile, expert-only) or user space (safe, but limited visibility, boundary-crossing overhead, can’t act in the fast path).
- The trade-off was stark — kernel power with kernel danger, or user-space safety with user-space limits — and it constrained what tooling was feasible for decades.
- eBPF is the missing third option: kernel-side power (runs in the kernel, at kernel events, with kernel visibility/speed) with user-space-like safety (the verifier guarantees no crash/invalid-memory/hang), loaded at runtime and (with CO-RE) portable across kernels.
- eBPF doesn’t tear down the boundary — it provides a safe, controlled gate for verified programs — which democratized safe kernel programming, removing the old trade-off and enabling the wave of eBPF observability/networking/security tools.