“We don’t need ChatGPT Enterprise. We already have Azure Foundry.”
“I can run a model locally, so my tokens are free.”
Both statements mistake access to one component for possession of the complete capability. When I evaluate an AI alternative, I want to know what work it enables us to complete, at what quality, with how much intervention, and at what total cost.
Getting a model to run is an engineering achievement. Making it a dependable working environment is a much larger undertaking.
Microsoft describes Foundry as a platform for building AI applications and agents. That makes it a legitimate option for constructing capabilities. It does not, by itself, establish that those capabilities already exist for the people who need to use them. Someone still has to assemble, integrate, evaluate, operate and maintain the resulting product.
That distinction is central to the case for complete AI products. The value includes the work users can perform immediately and the engineering they do not have to commission first.
The model and its environment work together
I have argued that the harness matters increasingly: the tools, context management, execution environment, skills and controls surrounding the model. But this should not be interpreted as “the model no longer matters.”
Models differ in their ability to plan, handle ambiguity, use tools and recover from mistakes. Training can explicitly develop these abilities. OpenAI’s Codex launch report describes reinforcement learning on real coding tasks across environments, including learning to run tests iteratively.
The model and its working environment therefore cannot be treated as entirely independent components. Reproducing a tool interface does not automatically reproduce the learned behaviour needed to use it effectively.
Research supports the importance of that interaction. SWE-agent showed that purpose-built interfaces improved agents’ ability to navigate repositories, edit code and execute tests.
In my own work, Codex and Claude Code have been substantially more effective than Copilot App with an arbitrarily selected model. That is my experience, not a universal benchmark. It is nevertheless why I assess the complete working system rather than assume that a model catalogue establishes equivalence.
Smaller models belong in the architecture
I already advocate using powerful models to define a solution and smaller models to implement clearly specified work.
In one of my own implementation experiments, a local model spent approximately an hour without producing a usable implementation. After a frontier model clarified the representation, algorithm and initial test sequence, the local model produced a working implementation. Tests and an independent reference comparison passed, although review still identified two small contract deviations.
The useful result was a division of labour. Frontier reasoning established direction; local execution performed much of the implementation.
Recent research describes a similar mechanism. The preprint Better Harnesses, Smaller Models found that adapting harnesses improved 16 of 21 task–model combinations, with seven closing the performance gap. Adaptation worked best for repetitive workflows and models with sufficient underlying capability.
Even Small Language Models are the Future of Agentic AI, a position paper strongly advocating smaller models, centres its argument on specialised, repetitive tasks and proposes heterogeneous systems where broader capabilities are required.
This is consistent with my approach: resolve ambiguity, capture decisions in durable artifacts, and delegate well-defined execution. It does not establish that every smaller model can independently perform the reasoning that made the delegation possible.
Local tokens still have a cost
Local inference can be economically attractive. Removing the provider’s token bill, however, leaves hardware, electricity, operation, integration and maintenance. It also leaves the time spent waiting, supervising, correcting and retrying.
Those costs should be measured on both sides. Hosted products also have limits, require review and can fail.
The relevant comparison is total cost per successfully completed and accepted task. AI Agents That Matter makes the underlying evaluation argument: accuracy and cost must be considered together, and model benchmarks should not be confused with evidence that an agent suits a particular application.
A cheap unsuccessful attempt is still an unsuccessful attempt. A more expensive system can be better value if it reliably returns useful capacity to its users.
Replaceability does not establish equivalence
This is a counterpoint to the previous pattern. Designing a system so that its model can be replaced does not establish that another model will perform the same work successfully.
My model-replaceability pattern argues that business rules, trusted data, workflows and evaluation criteria should remain under our control. We should be able to change the reasoning engine without surrendering the application’s intended behaviour.
Replaceability still requires a sufficiently capable replacement. It does not imply that every model, harness or product is equivalent. The mistake is treating the ability to swap a component as evidence that we have replaced the capability of the complete system.
I am open to local models, open weights, alternative vendors and systems built on Foundry. The decision should follow representative work: compare accepted results, elapsed time, human intervention and the full cost of providing the capability.
I’m open to replacing the vendor. Show me that we can replace the capability—and what it costs to do so.
