Stuff about Software Engineering

Month: May 2026

The Bottleneck Moved

For years, the dominant constraint in software engineering was implementation cost. Writing software was expensive. Changing it was expensive. The whole discipline organized itself around that fact — estimation, sprints, backlogs, delivery teams — because throughput was the bottleneck.

AI is dismantling that bottleneck faster than most organizations have been able to think about what replaces it.

But there’s a less-noticed shift happening alongside the capability story. Quietly, and then quite visibly, the major AI vendors are moving coding workflows toward consumption-based pricing. GitHub Copilot has introduced premium requests. Anthropic has repeatedly adjusted Claude Code access as demand exploded. OpenAI has separated agent-style workflows into explicit credit models for enterprise use. The details differ — different token pools, different rate limits, different structures — but the direction is consistent: frontier inference is not economically unlimited, and the vendors are starting to price it that way.

This matters more than it might appear, for one reason: AI-assisted development is not reducing demand for software. It’s accelerating it. Developers produce more. Non-developers can now produce some. Agents execute implementation work in parallel. If software was already eating the world, AI is increasing the bite rate.

Which means total inference demand is likely to rise faster than efficiency gains bring costs down. Even as models get cheaper per token, the sheer volume of generated code, tests, pipelines, reports, and scaffolding will grow. Organizations routing large amounts of that work through frontier reasoning models are building a cost structure that may become uncomfortable quickly.


The current pattern of AI-assisted development tends to look like this: open a long-running session with a flagship model, iterate continuously, keep large context windows alive, regenerate as needed. It works. The problem is what it’s spending expensive reasoning capacity on.

A significant portion of that work is not actually ambiguous. It’s repetitive. Deterministic. Structurally predictable. Generating a stable HTML report structure for the fifteenth time doesn’t require the same model that helped you design the architecture. Implementing a transformation you’ve already fully specified doesn’t require frontier-level reasoning. But because the workflow isn’t designed to distinguish between the two, it routes everything through the same expensive path.

That’s partly an economic inefficiency. But it’s also a signal. If implementation continuously requires frontier-level reasoning, that’s usually not a model problem — it’s a decomposition problem. It means ambiguity that should have been resolved upstream is still alive in the work.

Historically, you could absorb that ambiguity because implementation throughput was the real constraint anyway. When implementation gets cheap, the constraint moves. What’s left is ambiguity, architectural clarity, decomposition quality, validation, and organizational alignment. That’s where the work now actually is.


The architectural implication is straightforward, even if acting on it isn’t. Use stronger models where ambiguity and interpretation are genuinely high — architecture, requirements, tradeoff analysis, decomposition, synthesis. Once that work is done and structure has been established, the remaining implementation should be deterministic enough that it doesn’t need the same reasoning capacity.

I’ve been exploring what this looks like in practice through two experimental repositories, Rupify and Speckify. Rupify concentrates expensive reasoning at the front of a workflow: AI-assisted stakeholder interviews, requirements normalization, formal specification generation. The goal is to resolve ambiguity deliberately and early, producing canonical artifacts that downstream work can execute against. Speckify then takes those specifications and decomposes them into atomic, traceable implementation units — work that’s structured enough that smaller, cheaper models can handle it reliably.

The point isn’t “better AI coding.” It’s redesigning the workflow so that frontier reasoning gets used once, where it’s genuinely needed, rather than continuously throughout the entire delivery cycle. The economics look very different when you do that.


For a while, the dominant question in AI-assisted engineering was capability: which model, which agent, which coding assistant. That question isn’t going away. But another constraint is now becoming visible alongside it.

The organizations that scale this well probably won’t be the ones that put AI everywhere. They’ll be the ones that figure out where ambiguity actually lives in their workflows, structure everything else aggressively, and stop paying frontier-model prices for work that stopped being uncertain three steps ago.

The competitive advantage isn’t more AI. It’s better placement.

Pattern: Reserve AI for Ambiguity

A language model can interpret a request, calculate an answer, format a report and decide what happens next. That makes it tempting to give the model the whole workflow. But capability alone is a poor guide to architecture.

My starting point is simpler: reserve AI for ambiguity, and use systems for structure. Apply reasoning where it resolves uncertainty or improves understanding. Capture the parts that are already understood in code, rules, schemas and templates.

The cost problem is a design problem

Token budgets, usage caps and model tiers make inference costs visible. Yet repeated spending often points to something more fundamental: a workflow is asking the model to solve the same problem again.

Regenerating a stable report layout, applying established calculations or rediscovering a known mapping consumes reasoning capacity without creating new understanding. Increasing the budget may keep that workflow running, but it leaves the design problem intact.

I want the value of AI to accumulate. A resolved question should become a reusable decision, specification or component, rather than another prompt that must be interpreted on every execution.

Ask whether the step needs AI

Before choosing a model, I ask what remains uncertain in the task.

AI is well placed when the work involves interpreting incomplete information, exploring alternatives, synthesizing evidence, explaining findings or making contextual judgments. Those activities benefit from flexibility because the answer cannot be fully specified in advance.

Established calculations, repeated formatting, fixed routing and stable classification rules have a different character. If the expected behavior can be stated explicitly and tested, it should usually be implemented directly.

A high-frequency step with little ambiguity is a particularly strong candidate for removal from the model path. The question is whether each invocation adds useful reasoning or merely repeats settled work.

Structure the work before selecting the model

I use five steps to make that distinction practical:

  1. Define the structure. Clarify the inputs, outputs, constraints and boundaries.
  2. Capture repeatable logic. Put established rules into code, schemas, workflows or templates.
  3. Locate the ambiguity. Identify the interpretation, synthesis or judgment still required.
  4. Apply AI there. Give the model a bounded role with evidence and a clear way to assess the result.
  5. Choose the model last. Match its capability and cost to the reasoning that remains.

What this looks like in practice

Software engineering: use stronger reasoning to explore architecture, identify risks and clarify requirements. Once the design has been decomposed into clear implementation tasks, smaller coding models may be sufficient. A continuing need for frontier reasoning during routine implementation can indicate unresolved ambiguity upstream. I explore this further in Efficient Use of AI for Software Engineering.

Analytical reporting: compute facts, capture evidence, apply explicit rules and render the report through software. Use AI to explain findings and interpret their significance. The report should not depend on a model reinventing its calculations and layout each time.

Process automation: encode predictable steps in workflows and services. Use AI when the process encounters unfamiliar language, an exception or a decision that genuinely requires context. An agent can call dependable capabilities without having to reason through every operational detail.

Make placement a deliberate decision

This principle also changes how I assess AI adoption. More model calls do not necessarily mean more value. I look for useful reasoning, reusable outputs and less uncertainty in the work that follows.

Organizations need a shared understanding of where inference is justified, where explicit logic is preferable and how model tiers fit different tasks. Otherwise, scaling adoption can simply scale avoidable cost and variability.

The goal is to place AI where its flexibility matters—and preserve what it helps discover in forms that can be used reliably again.

Pattern: Efficient Use of AI for Software Engineering

Why this matters

AI-assisted development is moving to usage-based cost models where:

  • Cost scales with model choice and interaction patterns

  • Long-running sessions and iterative loops increase spend

  • Higher-capability models are significantly more expensive

At the same time, AI enables:

  • Rapid exploration of designs

  • Large-scale code generation

  • Parallel execution of implementation work

Without structure, this leads to:

  • Unpredictable cost

  • Inconsistent quality

  • Unnecessary rework

This pattern defines a structured way of working that maximizes value while controlling cost and complexity.


Core pattern

Use powerful models to define the solution. Use cheaper models to implement the solution.

The goal is to:

  • Concentrate reasoning effort once

  • Avoid repeated re-evaluation

  • Execute implementation in small, independent units


Step 1 — Use AI for solution design

Use the most capable model available to:

  • Explore solution options

  • Evaluate trade-offs

  • Validate architecture

  • Identify risks and edge cases

  • Define architecturally significant requirements (ASRs)

Output:

  • Solution design document

  • Clear constraints and assumptions

  • Agreed direction

This step should remove as much ambiguity as possible.


Step 2 — Make the solution executable

Translate the design into structured work:

  • Break into epics and issues

  • Define scope and expected outcome for each

  • Ensure each unit is:

    • Scoped

    • Unambiguous

    • Testable

Good decomposition is the primary control mechanism.

Well-defined units enable:

  • Predictable AI execution

  • Minimal context per task

  • Independent implementation


Step 3 — Execute with clean context

For each issue:

  • Start with a fresh context

  • Provide only:

    • The issue description

    • Relevant constraints

    • Local code context

Avoid:

  • Long-running chat sessions

  • Accumulated conversation history

  • Repeated "compaction" of context

Treat every task as a clean execution.


Step 4 — Use cheaper models for implementation

Once tasks are well-defined:

  • Use faster, lower-cost models

  • Focus on:

    • Implementation

    • Test generation

    • Applying patterns

If a task requires a high-end model:

The issue is likely underspecified.


Step 5 — Execute in parallel where possible

When issues are independent:

  • Use subagents or worktrees

  • Implement in parallel

  • Rely on:

    • Clear boundaries

    • Well-defined contracts

This enables:

  • Faster delivery

  • Better utilization of AI

  • Consistent results


Step 6 — Avoid long-context degradation

AI performance degrades in extended sessions:

  • Context grows

  • Signal-to-noise decreases

  • Output quality drops

Common anti-pattern:

  • Iterate continuously in one session

  • Compact context

  • Continue

This accumulates errors over time.

Recommended approach:

  • Keep interactions short

  • Reset frequently

  • Reintroduce clean inputs


Step 7 — Store state in artifacts

Do not rely on the model to maintain system state.

State should be captured in:

  • Design documents

  • Specifications

  • Epics and issues

This ensures:

  • Reproducibility

  • Consistency

  • Independence between tasks

The model executes — artifacts define the system.


Step 8 — Keep feedback loops controlled

Even with strong design:

  • Issues will evolve

  • Edge cases will appear

Handle this by:

  • Updating artifacts (not conversations)

  • Refining issues

  • Re-running tasks with clean context


Summary

This pattern enables:

  • Predictable cost

  • Consistent quality

  • Scalable execution

By:

  • Separating reasoning from implementation

  • Minimizing context

  • Structuring work into independent units

  • Executing with clean, repeatable inputs

Solve once. Structure clearly. Execute repeatedly.

© 2026 Peter Birkholm-Buch

Theme by Anders Noren — Up ↑