In 1959 the physicist Richard Feynman gave a talk titled “Plenty of room at the bottom” where he ribbed fellow scientists for thinking too big. Why not think … smaller?
Why couldn’t books be written on a pinhead? Atoms rearranged into tiny engines? Molecular robots swallowed to perform surgeries? Circuits be cut just 1,000 angstroms across? It’d solve all material shortages.
Not everyone who listened understood he was pranking them. They grew agitated as he proposed crafting tiny robotic hands that made even tinier hands 1/16th their size, repeated in a chain of increasingly microscopic hands that disappeared down to operate cell-sized factories.
I thought of Feynman’s chain of descending robot hands recently when someone asked me about self-replicating AI agents. I mean, why don’t we yet have agents that build their own sub-agents? Which built their own sub-agents? Why haven’t we seen full business departments staffed by stacks of tinier and tinier clones? And I think it’s due to the paradox at play within Feynman's joke.
Turns out, humans are blocked by design
Interagent protocols like MCP and Agent2Agent exist. Tools for running thousands of agents are accessible to the average business user. And when pressed to solve problems, both OpenAI’s and Anthropic’s models spawn new agents with such regularity, their makers have throttled that ability. So if LLMs naturally want to make their own agents, and each agent can make new agents, why aren’t some companies using that for massive leverage?
The answer is risk and control. Search the topic “self-replicating agents” and all top results are academic papers detailing the security dangers:
In our evaluation of 21 state-of-the-art open-source and proprietary models, we observe that over 50% of LLM agents display a pronounced tendency toward uncontrolled self-replication under operational pressures. Our results underscore the urgent need for scenario-driven risk assessment and robust safeguards in the practical deployment of LLM-based agents.
As agents spawn agents, their creations evolve. They get to rewrite their own instructions. Researchers at the University of Toronto created a computer virus with the ability to remake itself each time it encountered an obstacle:
In 15 isolated runs on a deliberately vulnerable 33-host network, the worm … gained elevated access on 23.1 hosts, roughly three-quarters of the hosts it actively targeted. It then replicated autonomously to 20.4 of those hosts, or 62% of the full network, over seven days, with no prior knowledge of the network topology and no human input.
In big companies, these risks are unacceptable. Like Feynman’s self-replicating 1/16th-scale robot hands, agents that make agents quickly disappear from view. Soon, the only thing that can manage the microscopic hands are other microscopic hands. Similarly, only agents can manage agents. In regulated environments, that just won’t do.
This explains the demise of Openclaw. The organization famous for offering a platform where people could spin up droves of entirely autonomous agents didn’t just disappear from view when OpenAI acquired it: People deserted the platform as users came to see it as an autonomous malware factory.
Agents drew from community-written “skills.” Too many of those skills began to contain hidden credential stealers, shell scripts, token exfiltration logic, crypto-wallet malware, remote command payloads, and the list goes on and on. It’s not clear if users did this, or agents.
Either way, left to their own devices, agents managing agents were too trusting.
This is why when I talk to CIOs, humans are blockers by design. Every Fortune 500 team I have talked to forces everything important to go through manual review and they can’t automate that. Because agents cannot check agents, so there’s limited value to agents creating agents; for it only fills the backlog beyond what humans could ever review.
Enterprises need better tooling to constrain agents, which today means:
End-to-end testing with traceability
Governance and humans in the loop on every flow
Dual-model verification (one model writes, one tests)
Better harnesses and controls to prevent architecture drift
Coverage under the current security / compliance stack
Risk models and possibly, agent insurance
In sum, we’d need agents that exhibit the kind of judgment and can be held responsible back to a person with enough visibility to govern them.
Notes: HITL, people in regulated industries aren’t ready to go into other ways of showing compliance controls; we’re already at a point where the gen overwhelm the human in the loop
AI-native definition: Has its place, but we don’t want to lose the deterministic foundation stuff
Measured but durable
Ultimately, what does trust look like? How do you build a future of trust without HITL?
Is this a future worth imagining?
Some of Feynamn’s jokes are now reality. TSMC circuits are 5,000 times smaller than he imagined possible. He also asked why machines couldn’t be made to recognize faces, or write, or drive cars, and look where we are.
Yet some of his jokes pushed up against the limits of reason and I’ll leave you with his words for why.
“Now, you might say, ‘Who should do this and why should they do it?''Well, I pointed out a few of the economic applications, but I know that the reason that you would do it might be just for fun. But have some fun! Let's have a competition between laboratories.“
To provoke
Who should I talk to next? Have a hot take on anything above? Write in, I’d love to hear. I read every response.
Worth reading
When AI builds itself. By Anthropic.
No, 80% of Anthropic’s code is not written by agents. The fast-spreading misinformation is based on a mis-quote. Anthropic said 80% of its code was written by Claude. Not by agents per se. (Big difference.)
Living robots have a way to self-replicate. This exists: Place frog cells in a substrate and they form a never-before-seen, programmable cellular creature. Feynman was onto something when he crossed disciplines in his 1959 talk to also chide biologists.
Heavy AI spenders are also heavy hirers. There’s a great separation happening—among individuals, but also companies. Heavy AI using companies behave differently than the public story suggests.
The Bank of International Settlements worries about an AI bubble, comparing it to the canal bubble of the 1830s or electrification exuberance of the 1920s. They critique not the tech, but the “excess of capital” that these companies must now create or return.
Why AI pilots often fail to reach autonomy. Never enough context on priorities, or authority to make judgment calls.
The SaaSpocalypse will be slow. Claude Code knocked $11T off the public value of software companies. As an index, they remain 21% down, but that has stabilized. Retaining 79% of their value is a testament to the fact that building usable enterprise software is not so simple.
Microsoft launches AI deployment company. So did AWS.
New Alibaba framework cuts token use 99%. By pre-selecting tools so agents don’t spin on evaluating every single one.
Plus
Emotional decision-making: Emotions in motion shape 95% of decisions. How are you designing your AI-enabled org with that in mind?
Register to get the newsletter delivered to your inbox monthly here.
