Engineering Leadership in the LLM Era — Part 1: The Expanding Control Surface

Something has been bothering me about the way we talk about AI in software engineering.

Most conversations are still about productivity.

How much faster can developers code?

Which coding agent is better?

How many pull requests can we generate?

How much implementation work can AI take over?

Those are useful questions.

But as an Engineering Manager, I think there is a more important one:

What happens to engineering leadership when execution itself becomes partially autonomous?

For years, our operating model was relatively familiar.

People designed. People implemented. Systems executed.

As Engineering Managers, we managed people, architecture, delivery, reliability, developer experience, and operational risk.

None of that is going away.

But another layer is appearing on top of it:

Agents. Context. Evaluations. Autonomy. AI cost. Verification. Governance.

This is what I think of as the Expanding Control Surface.

And I don’t think our leadership models—or our dashboards—have fully caught up yet.

When coding gets faster, the bottleneck moves

Spotify recently described this transition with a great phrase:

Coding is no longer the constraint.

Their engineers are increasingly working with AI coding tools, and the volume and speed of software creation are changing with them.¹

That sounds like great news.

And it is.

But removing one constraint usually reveals another.

If writing code gets much faster, review becomes more important.

If experimentation gets cheaper, evaluation becomes more important.

If agents can execute tasks, boundaries become more important.

If models can access internal systems, context and permissions become more important.

If teams can generate significantly more changes, our ability to verify those changes needs to grow as well.

In other words:

the bottleneck doesn’t disappear. It moves.

Google’s DORA research describes AI as an amplifier, and I think that framing is particularly useful for leaders.²

AI can amplify a strong engineering organization.

But it can also amplify weak testing, poor documentation, unclear ownership, slow feedback loops, and fragile architecture.²

The model may be the same.

The organizational outcome won’t be.

That’s why I don’t believe the old Engineering Management model is dead.

I think it is simply incomplete.

Our systems changed faster than our dashboards

There is an old engineering idea I keep coming back to:

You cannot manage what you cannot observe.

We already measure lead time, deployment frequency, MTTR, failure rate, throughput, availability, and many other useful indicators.

We should keep measuring them.

But AI introduces things those dashboards were never designed to see.

Imagine your team’s pull-request volume increases by 70%.

Great.

But what if senior engineers’ review load doubles?

Your AI agent successfully completes a task.

Great.

But what if it needed five human interventions to get there?

Your LLM costs $10,000 this month.

Is that good or bad?

Without knowing the cost per successful outcome, the number means very little.

Your AI feature has a 95% success rate.

Sounds excellent.

But what is inside the remaining 5%?

Formatting problems?

Incorrect tool calls?

Or a small number of dangerous hallucinations?

Average quality alone doesn’t tell us enough.

We increasingly need to understand the system’s failure topology: not only how often it fails, but how it fails.

There are other new signals too.

How often do agents need human help?

How often is incorrect behavior caused by the model versus outdated context?

How quickly can an agent learn whether its change was correct?

How much rework do AI-generated changes create?

Is AI actually freeing senior engineers for architecture and product thinking—or just giving them more code to review?

These aren’t vanity AI metrics.

They are signals about the health of a new engineering system.

The Generation–Verification Gap

There is one metric I think deserves special attention.

I call it the Generation–Verification Gap.

This isn’t an established industry metric. It’s simply a mental model I find useful.

Imagine AI increases your team’s ability to generate software changes by 4x.

More PRs.

More tests.

More experiments.

More migrations.

More refactoring.

But your ability to verify those changes grows only 1.5x.

Your reviewers are still human.

Your environments are still slow.

Your security checks are still partly manual.

Your test suite still takes twenty minutes.

Did your organization really become four times faster?

Probably not.

You created a verification deficit.

And eventually that deficit shows up somewhere:

review queues,

shallow approvals,

larger batches,

rework,

instability,

or incidents.

Meta’s work on risk-aware code changes and just-in-time testing is an interesting example of where this is heading: as change generation accelerates, verification systems also have to become smarter and faster.³⁴

This leads to a principle I think Engineering Managers should start paying attention to:

Every increase in generation velocity creates a corresponding demand for verification velocity.

If AI accelerates production, leadership has to accelerate validation.

Otherwise we’re not removing the bottleneck.

We’re just moving it downstream.

So what actually changed?

I think the shift can be summarized quite simply.

Yesterday, much of Engineering Management was about managing the execution of work.

Today, we increasingly need to manage the environment in which humans and machines execute together.

That means new questions.

How autonomous should an agent be?

What context does it need?

How do we define “good”?

How quickly can we verify what it creates?

How do we understand the real economics of AI?

And which decisions should always remain human?

These questions point toward a new set of leadership responsibilities.

Not a replacement for people leadership, architecture, DevOps, or engineering fundamentals.

An additional layer.

A New Leadership Stack.

And that’s what I want to explore in Part 2.

Because the answer to a larger control surface cannot simply be:

more managerial control.

It has to be something better.

References

@misc{bilgin2026engineering-leadership-i,
  title   = {Engineering Leadership in the LLM Era — Part 1: The Expanding Control Surface},
  author  = {M. Emre Bilgin},
  year    = {2026},
  url     = {https://mebilgin.com/posts/2026/08/engineering-leadership-i/}
}