Author: Peter Birkholm-Buch (Page 1 of 17)

We should not confuse agent orchestration capabilities with an Agent Orchestrator platform. The enduring architecture is simpler: expose enterprise capabilities through thin, governed APIs; let authorized AI harnesses discover and compose them; and treat the harness as the Agent Runtime rather than as a mandatory central platform.

That means:

• REST/OpenAPI for service contracts

• OAuth/OIDC for identity and authorization

• MCP or equivalent open mechanisms for machine-readable discovery where useful

• the harness for execution, skills, scheduling, state, policy and orchestration

The harness might be a coding assistant, a local application, a cloud service, a domain-specific runtime or a custom application. The capability should survive replacement of any one of them.

I’m not disputing the need for orchestration capabilities; I’m disputing the assumption that those capabilities imply an enterprise Agent Orchestrator platform.

The market framing

The market is converging around the idea that enterprises need an orchestration or control layer for AI agents. Gartner describes a fragmented market in which BOAT, BPA, AI agent platforms, iPaaS and open-source frameworks can all provide agent orchestration, and frames the emerging competition as one over control of enterprise execution:

• Choosing AI Agent Orchestrators: Apply a Structured Scoring Framework

• AI Providers That Automate Business Decisions Control the Future of Work

• Gartner research noting differing interpretations of AI agent orchestration

McKinsey uses the term agentic mesh for a composable, distributed and vendor-agnostic orchestration layer connecting agents and traditional systems. Its proposed capabilities include discovery, registries, authentication and authorization, observability, evaluations and governance. It also identifies repeatable and executable actions through secure APIs as foundational infrastructure for agentic systems:

• Rethinking enterprise architecture for the agentic era

• Seizing the agentic AI advantage

• Reimagining tech infrastructure for (and with) agentic AI

There is clearly a real set of capabilities here. The questionable step is the move from capability to mandatory central platform.

Old wine in new bottles

This closely resembles the integration-platform discussion. Integration platforms and iPaaS products solved real problems, but the architectural mistake was allowing the product to become the architecture, effectively making all integration depend on one central platform.

The same pattern is emerging again around agent orchestration. Workflow execution, tool invocation, authentication, state, scheduling, retries, audit, observability, policy and human approval are all legitimate needs, but most are familiar software and integration capabilities. The genuinely new part is that an LLM can decide dynamically what to do next, and multiple autonomous runtimes can collaborate. That creates a need for orchestration capabilities; it does not automatically create a need for a new enterprise platform category.

The harness is the Agent Runtime

A harness gives an AI model the ability to do useful work by combining instructions, skills, tools, context, permissions, execution semantics and guardrails. Increasingly, harnesses also provide scheduling, state, evaluation and observability. In practical terms, that is already an Agent Runtime.

It might be Codex, another local runtime, a cloud-hosted service, a scientific runtime or a custom application. There is no architectural reason why all of these must sit beneath one central Agent Orchestrator. The runtime is replaceable; the enterprise capabilities it consumes should be enduring.

Great APIs are the architecture

The durable enterprise asset is the set of business and technical capabilities exposed through stable service boundaries. Applied to AI, the implication is simple: if a capability can be securely invoked through a governed API, any suitable harness can orchestrate it.

The enterprise standard can therefore remain thin:

• standard web APIs, predominantly REST where appropriate

• OpenAPI or equivalent machine-readable contracts

• OAuth/OIDC authentication and authorization

• clear scopes and least privilege

• stable versioned contracts

• consistent auditability and policy enforcement

If business systems, data services and document workflows expose capabilities this way, a suitably capable and authorized harness can use them. A local Codex runtime executing skills on a schedule can orchestrate work. So can a cloud service, a domain-specific runtime or a traditional application.

Discovery does not require a central orchestrator

A predictable objection is that, without a central Agent Orchestration Platform, a runtime needs another way to know which capabilities exist. That is a discovery problem, not an execution-centralization problem.

UDDI, WSDL, OData and OpenAPI all attempted to make services describable and discoverable. Their weakness was that a developer still had to understand the interface and wire it into an application. AI changes that equation because a capable runtime can interpret machine-readable descriptions and decide dynamically how to use the advertised capability.

As described in Old Wine, New Bottles: Why MCP Might Succeed Where UDDI and OData Failed, MCP can connect services to AI runtimes capable of interpreting discovered capabilities rather than merely cataloguing them.

The resulting pattern is straightforward: REST/OpenAPI describes what can be called, OAuth/OIDC governs who may call it, MCP or equivalent mechanisms can help machines discover it, and the harness decides how to compose and execute it. Registries and catalogs may still be useful, but they should describe capabilities rather than become mandatory execution paths.

Governance belongs at durable boundaries

This approach does not weaken governance; it moves governance to places that survive technology replacement. At the capability boundary, authentication, authorization, policy, auditing and data contracts should be enforced. At the runtime boundary, the harness should govern models, skills, tools, execution, observability and human approval.

That lets us change the harness, model provider or execution environment without redesigning enterprise capabilities.

The test

If we replace the agent orchestrator in five years, do we still possess the orchestration capability? If not, we have created a platform dependency rather than an enterprise capability.

Keep the capability when the platform changes

There is a legitimate need for orchestration: scheduling, tool invocation, state, security, policy, observability, evaluation, human approval and coordination. We should resist turning those needs into a mandatory enterprise Agent Orchestrator.

Enterprise capabilities should be exposed through thin, governed standards, while autonomous runtimes and harnesses remain free to discover and compose them wherever they run.

I’m not disputing the need for orchestration capabilities; I’m disputing the assumption that those capabilities imply an enterprise Agent Orchestrator platform.

“We don’t need ChatGPT Enterprise. We already have Azure Foundry.”

“I can run a model locally, so my tokens are free.”

Both statements mistake access to one component for possession of the complete capability. When I evaluate an AI alternative, I want to know what work it enables us to complete, at what quality, with how much intervention, and at what total cost.

Getting a model to run is an engineering achievement. Making it a dependable working environment is a much larger undertaking.

Microsoft describes Foundry as a platform for building AI applications and agents. That makes it a legitimate option for constructing capabilities. It does not, by itself, establish that those capabilities already exist for the people who need to use them. Someone still has to assemble, integrate, evaluate, operate and maintain the resulting product.

That distinction is central to the case for complete AI products. The value includes the work users can perform immediately and the engineering they do not have to commission first.

The model and its environment work together

I have argued that the harness matters increasingly: the tools, context management, execution environment, skills and controls surrounding the model. But this should not be interpreted as “the model no longer matters.”

Models differ in their ability to plan, handle ambiguity, use tools and recover from mistakes. Training can explicitly develop these abilities. OpenAI’s Codex launch report describes reinforcement learning on real coding tasks across environments, including learning to run tests iteratively.

The model and its working environment therefore cannot be treated as entirely independent components. Reproducing a tool interface does not automatically reproduce the learned behaviour needed to use it effectively.

Research supports the importance of that interaction. SWE-agent showed that purpose-built interfaces improved agents’ ability to navigate repositories, edit code and execute tests.

In my own work, Codex and Claude Code have been substantially more effective than Copilot App with an arbitrarily selected model. That is my experience, not a universal benchmark. It is nevertheless why I assess the complete working system rather than assume that a model catalogue establishes equivalence.

Smaller models belong in the architecture

I already advocate using powerful models to define a solution and smaller models to implement clearly specified work.

In one of my own implementation experiments, a local model spent approximately an hour without producing a usable implementation. After a frontier model clarified the representation, algorithm and initial test sequence, the local model produced a working implementation. Tests and an independent reference comparison passed, although review still identified two small contract deviations.

The useful result was a division of labour. Frontier reasoning established direction; local execution performed much of the implementation.

Recent research describes a similar mechanism. The preprint Better Harnesses, Smaller Models found that adapting harnesses improved 16 of 21 task–model combinations, with seven closing the performance gap. Adaptation worked best for repetitive workflows and models with sufficient underlying capability.

Even Small Language Models are the Future of Agentic AI, a position paper strongly advocating smaller models, centres its argument on specialised, repetitive tasks and proposes heterogeneous systems where broader capabilities are required.

This is consistent with my approach: resolve ambiguity, capture decisions in durable artifacts, and delegate well-defined execution. It does not establish that every smaller model can independently perform the reasoning that made the delegation possible.

Local tokens still have a cost

Local inference can be economically attractive. Removing the provider’s token bill, however, leaves hardware, electricity, operation, integration and maintenance. It also leaves the time spent waiting, supervising, correcting and retrying.

Those costs should be measured on both sides. Hosted products also have limits, require review and can fail.

The relevant comparison is total cost per successfully completed and accepted task. AI Agents That Matter makes the underlying evaluation argument: accuracy and cost must be considered together, and model benchmarks should not be confused with evidence that an agent suits a particular application.

A cheap unsuccessful attempt is still an unsuccessful attempt. A more expensive system can be better value if it reliably returns useful capacity to its users.

Replaceability does not establish equivalence

This is a counterpoint to the previous pattern. Designing a system so that its model can be replaced does not establish that another model will perform the same work successfully.

My model-replaceability pattern argues that business rules, trusted data, workflows and evaluation criteria should remain under our control. We should be able to change the reasoning engine without surrendering the application’s intended behaviour.

Replaceability still requires a sufficiently capable replacement. It does not imply that every model, harness or product is equivalent. The mistake is treating the ability to swap a component as evidence that we have replaced the capability of the complete system.

I am open to local models, open weights, alternative vendors and systems built on Foundry. The decision should follow representative work: compare accepted results, elapsed time, human intervention and the full cost of providing the capability.

I’m open to replacing the vendor. Show me that we can replace the capability—and what it costs to do so.

For years, the dominant constraint in software engineering was implementation cost. Writing software was expensive. Changing it was expensive. The whole discipline organized itself around that fact — estimation, sprints, backlogs, delivery teams — because throughput was the bottleneck.

AI is dismantling that bottleneck faster than most organizations have been able to think about what replaces it.

But there’s a less-noticed shift happening alongside the capability story. Quietly, and then quite visibly, the major AI vendors are moving coding workflows toward consumption-based pricing. GitHub Copilot has introduced premium requests. Anthropic has repeatedly adjusted Claude Code access as demand exploded. OpenAI has separated agent-style workflows into explicit credit models for enterprise use. The details differ — different token pools, different rate limits, different structures — but the direction is consistent: frontier inference is not economically unlimited, and the vendors are starting to price it that way.

This matters more than it might appear, for one reason: AI-assisted development is not reducing demand for software. It’s accelerating it. Developers produce more. Non-developers can now produce some. Agents execute implementation work in parallel. If software was already eating the world, AI is increasing the bite rate.

Which means total inference demand is likely to rise faster than efficiency gains bring costs down. Even as models get cheaper per token, the sheer volume of generated code, tests, pipelines, reports, and scaffolding will grow. Organizations routing large amounts of that work through frontier reasoning models are building a cost structure that may become uncomfortable quickly.

The current pattern of AI-assisted development tends to look like this: open a long-running session with a flagship model, iterate continuously, keep large context windows alive, regenerate as needed. It works. The problem is what it’s spending expensive reasoning capacity on.

A significant portion of that work is not actually ambiguous. It’s repetitive. Deterministic. Structurally predictable. Generating a stable HTML report structure for the fifteenth time doesn’t require the same model that helped you design the architecture. Implementing a transformation you’ve already fully specified doesn’t require frontier-level reasoning. But because the workflow isn’t designed to distinguish between the two, it routes everything through the same expensive path.

That’s partly an economic inefficiency. But it’s also a signal. If implementation continuously requires frontier-level reasoning, that’s usually not a model problem — it’s a decomposition problem. It means ambiguity that should have been resolved upstream is still alive in the work.

Historically, you could absorb that ambiguity because implementation throughput was the real constraint anyway. When implementation gets cheap, the constraint moves. What’s left is ambiguity, architectural clarity, decomposition quality, validation, and organizational alignment. That’s where the work now actually is.

The architectural implication is straightforward, even if acting on it isn’t. Use stronger models where ambiguity and interpretation are genuinely high — architecture, requirements, tradeoff analysis, decomposition, synthesis. Once that work is done and structure has been established, the remaining implementation should be deterministic enough that it doesn’t need the same reasoning capacity.

I’ve been exploring what this looks like in practice through two experimental repositories, Rupify and Speckify. Rupify concentrates expensive reasoning at the front of a workflow: AI-assisted stakeholder interviews, requirements normalization, formal specification generation. The goal is to resolve ambiguity deliberately and early, producing canonical artifacts that downstream work can execute against. Speckify then takes those specifications and decomposes them into atomic, traceable implementation units — work that’s structured enough that smaller, cheaper models can handle it reliably.

The point isn’t “better AI coding.” It’s redesigning the workflow so that frontier reasoning gets used once, where it’s genuinely needed, rather than continuously throughout the entire delivery cycle. The economics look very different when you do that.

For a while, the dominant question in AI-assisted engineering was capability: which model, which agent, which coding assistant. That question isn’t going away. But another constraint is now becoming visible alongside it.

The organizations that scale this well probably won’t be the ones that put AI everywhere. They’ll be the ones that figure out where ambiguity actually lives in their workflows, structure everything else aggressively, and stop paying frontier-model prices for work that stopped being uncertain three steps ago.

The competitive advantage isn’t more AI. It’s better placement.