We're Measuring AI's Impact on Work at the Wrong Level

An article in MIT Technology Review caught my attention. It examines a problem economists are wrestling with as AI becomes capable of performing more of the tasks people do at work: How do we predict what happens to employment?

Economist Alex Imas argues that knowing which tasks AI can perform tells us surprisingly little about whether jobs will disappear. If AI enables a software developer to accomplish three days of work in one, an employer might need fewer developers. Or the company might produce more software, lower its prices, attract more customers, and ultimately need more developers.

Understanding how demand responds to changes in price is essential to making that prediction.

It's an important distinction, and it exposes something I've been thinking about for a long time.

We have built much of our understanding of work around people.

People have jobs. Jobs have responsibilities. Responsibilities can be broken down into tasks. We use those tasks to define roles, evaluate performance, estimate capacity, and determine how many people an organization needs.

That model has carried directly into how we're developing and applying AI.

We identify tasks that AI can perform. We build tools to automate them. We create agents that can execute increasingly complex tasks, then orchestrate those agents to complete entire workflows.

We measure the resulting productivity improvements and begin calculating how much human labor might be displaced.

There's a reasonable logic to all of this. But it rests on an incomplete understanding of how organizations produce outcomes.

Work happens within systems

Consider a company developing software.

Its developers write code, but delivering software involves much more than writing code. Someone has to decide what to build, establish priorities, resolve competing requirements, coordinate dependencies, approve changes, manage releases, and respond when something goes wrong.

These activities involve people, technology, decisions, and constraints interacting across the organization.

Now imagine introducing AI that makes every developer three times as productive.

The developers can produce substantially more code. But product decisions still take weeks. Teams still have conflicting priorities. Changes still wait for approvals. Dependencies still require coordination across departments.

The company has dramatically increased its capacity to produce code without necessarily increasing its capacity to deliver software.

In fact, the additional output may create more work for the people and systems responsible for reviewing, integrating, testing, and releasing it.

The organization has become busier, and perhaps more expensive to coordinate, without a corresponding improvement in outcomes.

This is possible because individual productivity and organizational effectiveness are different properties.

An organization is a coordination system. Its outcomes emerge from the interactions among its participants, including the technology they use and the structures governing their work.

People are essential participants, but their individual capabilities don't determine what the system can accomplish on their own.

We've carried the same assumption into AI

What interests me most about the current AI industry is how closely its development follows our existing model of work.

We started by building tools that help people perform tasks. Then we began building agents capable of performing those tasks independently. Now we're developing increasingly sophisticated ways to orchestrate multiple agents, assign work, manage dependencies, and coordinate their execution.

In doing so, we're recreating many of the structures we already use to organize human work.

But automating a workflow doesn't establish that the workflow is well designed. Making decisions faster doesn't establish that authority is appropriately distributed. Increasing the volume of completed tasks doesn't establish that those tasks are moving the organization toward its intended outcomes.

An AI agent can execute its assigned responsibilities perfectly while contributing to a system that consistently fails to produce what it's supposed to produce.

The same has always been true of people.

This is why the distinction matters so much. We've spent decades trying to improve organizational performance by improving the performance of the people within organizations. AI now gives us an extraordinary ability to accelerate that same approach.

And acceleration changes the pressure on the system.

A decision bottleneck that was manageable when ten requests arrived each day may become a serious constraint when a hundred arrive. Dependencies that were coordinated informally may become impossible to manage at higher volumes. Failures that were previously caught by experienced employees may propagate before anyone recognizes what's happening.

AI can expose these weaknesses, amplify them, and introduce new ones.

A different unit of analysis

Organizations are coordination systems.

Within them, people, teams, software, and AI agents participate in moving work from intent toward an outcome. Each participant operates within a structure that determines what decisions it can make, what information it can access, what constraints govern its behavior, and how the system responds to change or failure.

Those relationships matter as much as the capabilities of the participants.

And because a team can itself be understood as a system, just as a company can, the same principles apply at different levels. A system can participate as an Actor within a larger system.

This changes how we examine productivity.

Instead of beginning with the tasks a person or AI agent can perform, we can begin with what the system is supposed to produce. We can follow work through the system, examine how its state changes, identify where decisions occur, and determine what constrains progress toward the intended outcome.

From there, we can examine what happens when a participant becomes more capable, when a task is automated, or when AI assumes responsibilities previously performed by people.

The consequences depend on the structure of the system receiving that new capability.

What are we really measuring?

The MIT Technology Review article raises an important economic question. Even when AI improves productivity, employment outcomes depend partly on how demand responds.

But there's another relationship worth examining before those productivity gains ever reach the market.

An improvement in the productivity of a task doesn't guarantee a corresponding improvement in the productivity of the organization.

That distinction has implications for how we evaluate AI investments, redesign jobs, introduce agents, and anticipate changes in employment.

It also challenges the way we think about organizational performance more broadly.

We've become very good at measuring the activity of the people who work inside organizations. AI is making us even better at measuring and accelerating the activity of artificial participants.

We remain remarkably poor at understanding the systems that turn all that activity into outcomes.

Until we address that, we'll continue to mistake increased capability for increased effectiveness.

And AI is going to make that mistake a lot more expensive.

Inspired by The one piece of data that could actually shed light on your job and AI, MIT Technology Review, April 6, 2026.