Principles of Agent Factories
Software used to be a strategic asset. It isn’t anymore. Code is cheap to produce, and getting cheaper by the month. Software is close to being a commodity.
AI and AI agents are solely responsible for this. And the next step in this AI revolution is already here: agent factories, or software factories.
Factory raised $150m at a $1.5 billion valuation. Mastra launched an AI software factory. Gastown reached 17k GitHub stars (as of the date of publishing). Google launched Antigravity, an AI-native IDE built to manage agents instead of code. And smaller tools like Agent Orchestrator appear by the week.
An agent factory is nothing more than a whole fleet of agents that work towards a single goal (often implementing a new feature in a software product), without much babysitting. A single developer is not interacting with a single agent harness anymore; they’re now managing a whole fleet, sometimes up to 100 agents, at the same time. Factories are going to help you build, run and scale hundreds or thousands of these AI agents.
It’s not you spinning up an agent anymore; it’s another agent that does that. And it does this at a scale unmanageable by a person without a solid system.
The irony is that building an agent factory in the technical sense is not the hard part; you can use AI agents for this. The hard part is building a reliable, cost-effective agent that produces consistent, quality output and is easy to upgrade once newer models are released.
How?
How do you create a factory? An agent factory is a set of principles, not a platform or tool you buy. No single architecture is going to win; the simpler the architecture, the more likely it is to succeed.
In the most lightweight form I see directories with preconfigured agents, including skills, MCP servers, Dockerfiles and AGENT.md for each role. Maybe even with CLIs and environment variables provisioned. So an agent configuration is just a plain folder on a filesystem.
To make agents act responsibly and predictably, and keep them easy to debug and improve, we need a few principles in place. They fall into two groups — running the fleet, and keeping it under control — plus one discipline that tells you whether any of it works: benchmarking.
Orchestration
If you want to run a whole fleet of agents, the first problem you run into is that managing all these agents by yourself is going to be challenging and mentally exhausting. This is where orchestration comes in. Orchestration is like the HR department of your factory.
In the non-AI world this is more or less a solved problem — we have Kubernetes and Temporal. The job description is clear enough: the orchestrator spawns agents, checks whether they are still working on their task, and spawns a subagent when an agent needs to do more in-depth research.
If you have agents that can spawn other agents, you need a few rules they live by — otherwise you end up with low quality output, or tokens burned on irrelevant or outdated tasks.
The simplest model is a single orchestrator spawns agents, the agents deliver the work, and a merge queue handles merging data, code and documents.
Orchestration is an elevated privilege; not every agent should be able to spawn any other agent. Some agents should just be workers; others can spawn very specific agents in specific scenarios - these are like managers. Other agents, like a chief of staff or recruiter, exist solely to spawn other agents.
Concurrency
The point of an agent factory is not cost savings; it’s speed.
If you run agents in sequence you are not gaining much speed. We gain the most by running agents in parallel. But parallelism has a cost: concurrently running agents are not aware of what other agents are doing at the same moment, so you risk conflicts and wasted tokens.
The fix is not to make agents coordinate with each other; it’s to make sure they do not have to. Give each agent its own copy of the world and merge the results at the end — which is why isolation, further down, is not optional.
Observability
Speed is the goal, but the factory still needs to produce more value than it costs. Spinning up 1,000 agents and burning through tokens is the easiest thing in the world. You want to see how many tokens your factory is burning as a whole, per task, but also per role.
To get this, you need full observability of each agent session. Without observability, there’s no cost control. But more importantly: there’s also no optimization path. You need to be able to evaluate each session, with its full context, to understand where things went wrong.
Isolation
Another principle I strongly believe in is that each agent needs an isolated environment, with no real time dependencies like a centralized database or issue management system. It removes any tight coupling with centralized memory management - yet it gives you the ability to give the agent a rich context it can utilize at runtime.
Every agent should be sandboxed by default in my opinion. Not sandboxing an agent means you are just rolling the dice and hoping you do not get compromised through a prompt injection. I doesn’t matter if you use commercial services like E2B, Exe, Daytona, or set up an open-source solution like Docker Sandbox or OpenSandbox. Any one of these solutions lets you roll.
Sandboxing alone is not sufficient. A sandbox gives you control over the environment — egress control comes out of the box — but it says nothing about the data your agent can reach. If you have designer, DevOps and developer agents, chances are not every one of them needs access to deploy to production. Just like in any organization, access should follow the role.
There is no standard yet for managing memory and context in agent factories. What is clear: any memory or context your agent can reach has a direct impact on the quality of its output, so curate it with care.
Human in the loop
Every factory should be able to run in two modes: HITL mode, where the human has a few pre-defined intervention points — like specification and review — and Auto mode, where the factory runs fully autonomously.
For HITL mode to work, each agent needs a channel to reach a human; whether that’s Slack, email, Telegram or something else does not matter. The easiest way to get this is the hooks API most coding harnesses ship with, but you can roll your own if you like.
The catch with human intervention is that approval can take hours, or worst case days, to arrive. That is a long window for timeouts, crashes and other agent failures. So it should be possible to pause an agent session while it sits idle and resume it later, with the guarantee that it picks up exactly where it left off.
Benchmarking & evals
It should not matter whether you use Claude Code, Codex, or a framework like LangChain to build your agents, or whether you roll your own harness. The principles here apply to all of them. This matters more than it seems: models leapfrog each other every few months, and a factory welded to one harness ages with it.
But if the harness and even the provider do not matter, then what does matter is how you evaluate your agents over time. This is where evals and benchmarking come in. You need a semi-scientific method to gauge whether a newly released model actually suits your agent better, or whether you should stick with your current configuration.
The hardest part about the benchmark is that it should reflect actual quality AND should be expressible as a number. Collect real cases for your benchmark, not generated test cases. Pull 20-50 actual cases. Especially include the hard edge cases, not just the simple and plain ones. Decide what counts as a pass or fail.
Do not treat the benchmark as a strict rule. You probably do not have the resources to run it thousands of times and get significant results.
The durable part of a factory is not the agents; it is everything around them. Your orchestration rules, your curated memory, the benchmarks you have built, your isolation and access model. None of that can be easily vibe coded. And it compounds: every session you evaluate, every intervention point you tune, every agent you develop by hand makes your factory a little better at producing quality output.
That’s also why I don’t think a single platform wins here. A platform can sell you the machinery, not the operating discipline. The principles are the factory.
So don’t start by spinning up a hundred agents. Start with one. Develop it by hand, push it to the max until you feel its limitations. Build a tiny benchmark from twenty real cases. Wrap it in orchestration, observability and isolation. Then add the next one.
A factory that scales bad agents just produces garbage faster.