Models

smolagents agents write code, so the model behind them is what determines how well they work. This post covers models in smolagents: its model-agnostic design (Hugging Face models, hosted APIs, local models, and more), why model choice is especially consequential for code agents, and how the code-agent approach interacts with the model landscape. The library’s Hugging Face origin makes open and local models first-class, which is part of its appeal.

Model-agnostic, with open models first-class

smolagents is model-agnostic — it works with many model backends through its integrations, and you choose the model when creating the agent:

• Hugging Face models — naturally, given the library’s origin, models from the Hugging Face ecosystem (via the Hub and inference options) are first-class, including open-weight models.
• Hosted API models — proprietary models from providers (via integrations like LiteLLM and provider APIs), so you can use frontier hosted models too.
• Local models — you can run models locally (e.g. via local inference servers), which matters for privacy, cost, and offline use — connecting to the on-device / local inference and LLM serving themes.

This breadth — open, hosted, and local models all supported — reflects both the Hugging Face ethos (open models are first-class, not an afterthought) and the general keep-the-model-swappable principle. You write the agent once and choose (and change) the model behind it. The first-class support for open and local models is a distinguishing feature: where some frameworks orient around hosted APIs, smolagents makes running an open model (including locally) straightforward, which suits privacy-sensitive, cost-sensitive, or offline scenarios.

Why model choice matters especially for code agents

Model choice matters in any agent framework, but for a code agent there’s a specific dependency: the model has to write good code. The whole code-agent advantage (the earlier posts) rests on the model producing correct, expressive code as its actions — so the model’s code-writing ability directly determines the agent’s capability:

• The agent’s actions are code the model writes — so a model that writes better code produces better actions, and a model that writes buggy code produces failing actions. Code-writing quality is action quality in a code agent.
• Code fluency is the relevant capability — for a code agent, you care specifically about how well the model writes code that correctly calls tools and composes logic, which is a somewhat different (and often strong) capability than general chat. The good news, from the code-agent posts, is that models are generally very good at code (trained on vast amounts of it), which is part of why code agents work.
• Weaker models can still write usable code — because code-writing is so in-distribution for LLMs, even smaller models can often produce workable code actions, which is part of what makes smolagents practical with open and local models. You don’t necessarily need a frontier model for a code agent to function, though a more capable model writes better, more reliable code.

So for a code agent, “choose a model that writes good code” is the specific selection criterion, and it’s why smolagents pairs naturally with the many capable open code-writing models. The model isn’t just a component — it’s the author of every action, so its code ability is the agent’s ceiling.

Model choice as a cost, privacy, and capability lever

Because the model is swappable and central, it’s a lever for the familiar production concerns, with smolagents’s open/local support adding a privacy dimension:

• Cost — the cost-playbook lessons apply: choose the model that minimizes cost per completed task (and note code agents’ efficiency — ~30% fewer LLM calls, from the earlier post — already helps), and consider that open/local models can shift cost from per-token API fees to your own compute (the managed-vs-self-host trade-off).
• Privacy — running an open model locally means the agent’s inputs and code execution stay on your infrastructure, not sent to a hosted API — a real advantage for sensitive data (the local-first theme from the on-device series). smolagents’s first-class local support makes this practical.
• Capability — swap in a more capable code-writing model for harder tasks, or a smaller/local one where it suffices, matching model to task; as better open code models arrive, adopt them by configuration.
• Offline / control — local models let a code agent run without external dependencies, useful for controlled or disconnected environments.

The through-line: smolagents’s model-agnosticism, with open and local models first-class, gives you unusually broad choice — from a frontier hosted API for maximum code quality to a local open model for privacy and cost — and model choice matters especially because the model authors the code that is the agent’s actions. Choose the model that writes good code for your task, and use the open/local options when privacy, cost, or control call for them. The next post covers how the agent loop runs and how multiple agents compose.

Key takeaways

• smolagents is model-agnostic with open models first-class (its Hugging Face origin): Hugging Face/open-weight models, hosted API models (via integrations like LiteLLM), and local models are all supported, so you write the agent once and choose/change the model behind it.
• First-class support for open and local models distinguishes it — where some frameworks orient around hosted APIs, smolagents makes running open models (including locally) straightforward, suiting privacy-sensitive, cost-sensitive, and offline scenarios.
• Model choice matters especially for code agents because the model writes the code that is the agent’s actions — code-writing quality is action quality, so “choose a model that writes good code” is the specific selection criterion.
• Code-writing is highly in-distribution for LLMs (trained on vast code), so even smaller/open models often produce usable code actions — making smolagents practical with open and local models — though a more capable model writes better, more reliable code.
• The model is a cost lever (cheapest-per-task, aided by code agents’ ~30% fewer calls), a privacy lever (run open models locally so data stays on your infrastructure), and a capability lever (swap for harder tasks, adopt better open code models by configuration) — broad choice from frontier hosted to local open.

Further reading

• Tools (previous post)
• smolagents documentation — models
• On-Device AI with Gemma and Flutter — local model inference