At the Fortune 500 companies I am talking to, AI has already reached the core. That isn’t what I expected, but it’s what I’m finding.
I recently joined Lensing and set off on a listening tour where I talked to dozens of CIOs. Speaking off the record, they say AI is already essential to how their teams develop software. And they’re less focused on the differences between specific models than they are concerned about issues like context poisoning, role-based access, and development lifecycles. They’re talking about ‘portable supercontext’ and debating whether one can measure code velocity in customer Net Promoter Score™ (NPS).
In this inaugural issue of my new newsletter, Aperture, I summarize what I heard these last few months. Including, at the end, a shortlist of metrics those CIOs are using to define AI success. My hope is that this helps frame your thinking, as it has mine.
This newsletter is meant to be collaborative—if this sparks an idea for you, I’d love to hear it.
For sophisticated orgs, vibe coding has its limits
The CIOs I talked to are in aerospace, manufacturing, and banking, where the audit chain is non-negotiable. They need immutable documentation to audit every agent’s rationale and for whatever they build to survive exhaustive critical design reviews.
One exec put it this way: They can’t build things to work—they must build them to not fail.
That’s why I was surprised to find that regulated enterprises seem to have gone the furthest I’ve seen with AI. They already use it in critical workflows. AI is operating at the core.
Here is, briefly, what I heard.
// They use AI to help generate production-ready code
AI is writing specs, reviewing PRs, scanning documentation for compliance, generating software bill of materials (SBOMs), regenerating legacy code bases, and creating ephemeral, project-specific apps. AI is not doing this autonomously, however, as we will discuss.
// Everyone wants ‘portable supercontext’
In the enterprise, context matters more. Perhaps the most. From what I can see, it’s a greater bottleneck than access to compute. And not just storing context, but making use of it, because more is not always better. Some CIOs warned they were running into ‘context poisoning’: "There's only so much you can stuff before it gets confused. The winning architecture is an LLM stack tuned with weights against an enterprise data set, not a bigger prompt.”
I cannot stress this enough: AI that doesn’t retain context on the enterprise software development lifecycles only creates more work. Said one friend, “It generates without reference to architecture … agents build whatever and take functionality out of the box.”
// Software development is different now
CIOs want more action from AI vendors. “Stop scaring me about AI code in production,” said one. “That moment has passed. Help me document and design and trace.” They also want help doing more than generating code. That’s just one step.
Here’s a change I’m seeing that I agree with: The new consensus is that code and spec should be one and ship together. Code edits should update docs, and vice versa; two halves of one self-healing record.
“Most developers have never liked writing documentation,” lamented one CIO. AI may be the only way to keep up.
// The SaaSpocalypse is coming and may be uneventful
Everyone I talked to is aggressively building applications they used to buy. “All software is being evaluated now,” said one. These IT orgs are reviewing their 1,000s of applications with a retire/replatform/rebuild framework and a hatchet. Any SaaS vendor that doesn't seem to be taking AI seriously risks looking obsolete. CIOs are okay with some feature loss if they can consolidate.
However, this rapid creative destruction may be, like all creative destructions, net-net. Companies seem to be reinvesting any savings in other, more AI-forward software, or just tokens. CIOs aren’t yet clear on the cost savings. Which brings me to my next point.
// Core systems matter more
No one is considering replacing their core business applications: ERP, CRM, HCM, PLM, etc. “You can’t pull out the plumbing,” said one. But they are experimenting aggressively with custom billing automation, configurators, and that whole constellation of supporting apps that collectively cost $10 to every $1 spent on the core system.
// People and processes are still a sticking point
Old org structures can’t do new tricks. “Everybody can do everything with AI in theory … but organizations haven’t. We still have legacy PMOs and scrum masters, and no platform addresses this shift,” said one CIO. Some are ‘shifting left’ for governance by asking experts to get involved earlier in approvals.
"We've seen faster development,” says one. “We've also seen more defects going to the testing phase. I don't think we've put our finger on it just yet."
// Humans retain a chokehold by design
At every organization, humans are still in every loop, which is both limiting and essential. “If AI generates 20,000 pages of requirements from a legacy app migration, no human can absorb all that,” said one. But slow is preferable in an industry regulated by FINRA, whose fines can stretch to nine figures. “Human in the loop is non-negotiable for prod-bound code.”
“Human in the loop is non-negotiable for prod-bound code.”
// Cloud costs are a real concern
Some are building hybrid infrastructure of cloud plus private cloud closer to the GPU to control runaway costs. All are encouraging their people not to use AI for its own sake. "There needs to be a business-level decision as to the reason why you're using this, and it's not just because it's a new fancy tool,” said one. “AI is a multiplier, not a justifier.”
“AI is a multiplier, not a justifier.”
// They don’t want lower headcount—they want higher NPS
Every tech team’s CFO wants to know how much headcount they can save. But CIOs don’t want fewer engineers. They want to slash systems integrator contracts: “The ROI isn’t 2x developers. It’s 20% less SI spend,” said one. And to cut out Saas: “The value for us now isn't doing software development. The value for us is getting rid of software vendors.”
CEOs want to know how much more efficient developers have become. But one CIO was adamant that “developer productivity is very personal … one person is 20% more, one person is 30% more … I prefer team-level metrics and time-to-market math.”
One bank CIO is adamant that the only way to measure AI’s impact is by the same way the bank measured it before AI: feature velocity and NPS.
Many can’t hire developers fast enough. But they’re getting blocked, because everyone has a sense that IT orgs should be doing more with less: “To support any additional devs, I need more UX people and tech writers, but the company doesn’t want to pay for it.”
Metrics I heard mentioned:
Net Promoter Score™ (NPS)
Project / feature velocity
Reduced Saas spend
Reduced systems integrator spend
“The ROI isn’t 2x developers. It’s 20% less SI spend [and] getting rid of software vendors.”
What does it all net out to?
Every CIO I talked to had several stories of definitive, time-saving AI wins. Here’s one of my favorites: An insurance company had a 40-year-old mainframe reporting application that it wanted to replace. They had hired an offshore team, and after two years, that team had only documented 20% of the code. So their onshore team used Claude Code to completely replicate the software in a newer language … in two days.
Limitations abound, but take my word for it: AI has reached the core. In the next issue of this newsletter, I tackle the next question you’re probably wondering: How far away are truly autonomous agents?
To provoke
Who should I talk to next? Have a hot take on anything above? Write in, I’d love to hear. I read every response.
Worth reading
DM Radio: Technical debt—what can be done? A peek at how I’m thinking about the technical debt issue in the enterprise. AI code-gen is mature. But that’s just one step in the lifecycle. (Listen, 27 min)
CFO.com: Who defines AI’s value? 60% of finance teams think they should be involved. Only 26% say they are. (Read, 6 min)
HBR: You shouldn’t treat AI as employees. A useful discussion of what we risk when we give agents a name and photo. A study shows people are 8% less likely to question an anthropomorphised agent. (Read, 9 min)
FT: How much value is AI actually creating? Few apps have gained traction. Supports the conversations cited above. (Read, 5 min; paywall)
Diginomica: Tokenomics. Tokens are not results. Some large orgs are demanding a different model, like agentic work units.
Plus
Peta Pixel: the comet. For two years, the European Space Agency’s satellite followed an asteroid, 67p, moving at 84,000 miles per hour. It took 400,000 photos. A motion designer and composer created a three-minute video to give a sense of what it feels like to exist on that comet. (Watch, 3 min)
