The Wrong Layer of Abstraction

AI was never directly coming for our jobs in the way people imagined, because AI exists at the wrong layer of abstraction.

In February, Mustafa Suleyman, the CEO of Microsoft AI, told the Financial Times that most of the tasks accountants, lawyers, and other professionals perform will be fully automated within 12 to 18 months. It was the most aggressive version yet of a prediction the industry has been making since late 2022, when ChatGPT pushed transformer-based AI into the public consciousness: AI can write, code, summarize, analyze, and generate deliverables, so AI is coming for the jobs built around those deliverables.

In my last piece, I pushed back on Nikita Bier’s claim that AI ended technical moats. Bier was right about which moats matter and myopic about the timeline. Suleyman’s prediction has a similar shape. He may well be right about the tasks. Where I think he’s wrong is in assuming that automating the tasks means automating the jobs.

It’s also worth noting where the prediction comes from. Suleyman made it while explaining how Microsoft expects to sell “professional-grade AGI” into the enterprise, and from Microsoft’s seat, he may be closer to right than most. Microsoft’s moat was never taste. It’s distribution, enterprise contracts, and an ecosystem most companies can’t easily leave, the network-effect moats I wrote about last time. A company like that can afford to let its judgment go generic, because customers aren’t buying its judgment. Most companies aren’t Microsoft.

Notice the word: tasks. Almost the entire debate about AI and work has been about capability. Can it write production code? Can it do the analysis? Can it build the deck? Every model release moves that line, and every argument built on it has a shelf life. But can doesn’t mean should. Whether AI can do the work matters a lot less than whether a business should let AI own it.

AI was never directly coming for our jobs in the way people imagined, because AI exists at the wrong layer of abstraction.

The deliverable was never the whole job. In most professional work, the deliverable is just the visible output at the end of a much larger process. Before anything ships, there are alignment checks, quality checks, revisions, liability reviews, stakeholder approvals, and some form of acceptance testing. And before even that, there are upstream decisions: cost analysis, competitive positioning, technical feasibility, branding cycles, customer research, roadmap tradeoffs, organizational politics, and basic questions around whether the work should exist in the first place.

AI sits squarely in the middle of that chain. It can accelerate artifact production. It can write the memo, draft the code, summarize the meeting, generate the deck, scaffold the design, and produce the first-pass analysis. That’s real, and it matters. But it doesn’t automatically know what work should exist, who it’s really for, why it matters, what risk it creates, who needs to accept it, or whether the final result is actually good enough. It doesn’t own the judgment, accountability, context, taste, liability, or strategic alignment that make the artifact valuable.

Professional work isn’t just deliverable production. It’s deciding what matters, shaping it through constraints, navigating stakeholders, absorbing risk, and getting other humans to accept an outcome. As long as humans sit upstream approving direction and downstream accepting outcomes and absorbing liability, AI isn’t running that process. It’s accelerating one part of it.

The Downstream Problem

Even if AI speeds up the middle layer, we’re still fundamentally constrained by upstream and downstream processes. If a business can suddenly generate more code, more documents, more designs, more analyses, more proposals, more campaigns, and more strategy decks, then the business also needs more judgment around what should be generated, what should be prioritized, what should be reviewed, and what should actually ship. Otherwise, the company hasn’t created velocity. It’s created noise.

Open source got an early look at this. Daniel Stenberg, who maintains curl, has spent the last couple of years publicly fighting a flood of AI-generated security reports. They look like legitimate vulnerability disclosures: properly formatted, confidently written, and full of technical detail. Most of them describe bugs that don’t exist. Generating one costs the submitter next to nothing, while triaging one costs a maintainer hours. Look at where the cost landed. AI made producing the report free, left the judgment exactly as expensive as it was, and then multiplied how much judgment was needed.

The same thing happens inside companies, just more quietly. A company can drown itself in plausible outputs: more drafts, more pull requests, more “strategic options,” more AI-generated analysis, more work that looks done but still needs a competent person to decide whether it’s correct, useful, aligned, safe, or worth doing at all. This is the downstream problem. AI can multiply output faster than organizations can multiply trust.

Businesses that want real AI velocity increases will either need to increase upstream and downstream capacity, or rely much more heavily on business taste makers: the people who can decide what matters, what’s good, what’s dangerous, what’s on-brand, what will actually sell, what will survive review, and what should be killed. Either one is a hard pill to swallow in the current corporate paradigm. Much of middle management’s job is approving deliverables, and with corporate America’s obsession with reducing middle management, companies will need to give more power, and more diverse power, to lower positions such as team and department leads, or manager-level ICs.

Many companies want AI to reduce headcount, flatten expertise, and make execution cheaper. But if AI increases the amount of output moving through the system, then the people who can judge, filter, direct, and accept that output become more important, not less. The more the middle layer accelerates, the more pressure gets pushed to the edges. AI doesn’t eliminate judgment. It creates more stuff that has to be judged.

Deliverables Are Not Finite

There’s also a deeper issue with the idea that AI simply reduces the need for engineers or other white-collar workers: deliverables are not finite.

In 1865, the economist William Stanley Jevons noticed something counterintuitive about coal. More efficient steam engines should have reduced Britain’s coal consumption. Instead, consumption went up, because cheaper power became worth using in places it never had been before.

Software has followed the same curve. Each higher-level abstraction in engineering has led to more engineering output, not less. Compilers didn’t reduce the demand for software, and neither did frameworks, cloud platforms, or SaaS. They increased what could be built, who could build it, and the number of problems that became economically worth solving. Jevons isn’t a law of nature. It holds as long as there are more problems worth solving than people to solve them, and for software that’s been true for fifty years.

There’s no fixed amount of software waiting to be written. When software gets cheaper to produce, companies don’t simply stop hiring engineers. They build more internal tools, more integrations, more workflows, more dashboards, more automations, more experiments, more compliance systems, more customer-specific features, more niche products, and more software for problems that were previously too expensive to justify.

This doesn’t mean every engineer is safe. It doesn’t mean there will be no displacement. It doesn’t mean every current job survives in its current form. It means the simplistic model of “AI writes code, therefore fewer engineers” misses how software actually expands.

The bar for useful and profitable software has gotten higher since the dot-com era. “Throw it on the web and make millions” is mostly over. But there’s still a huge amount of opportunity in niche industry software, internal operations, compliance automation, healthcare workflows, logistics, manufacturing, government contracting, financial infrastructure, and adapting software to specific foreign markets and business cultures.

AI will raise the floor. The easy deliverables will get cheaper, and mediocre output will get harder to justify. If the only thing someone does is produce decent-looking work from clear instructions, that layer gets exposed pretty quickly.

But technical depth will also increase. Novel tools will be created by AI-enabled engineers. Smaller teams will attempt more complex systems. More domain-specific software will become viable. More industries that were too fragmented, boring, regulated, or operationally weird for traditional SaaS will suddenly become worth serving.

AI Cannot Own The Value Chain

Even if we could automate some of the upstream and downstream decision making, there’s a limit, and it has less to do with capability than with what customers actually buy.

In that same piece, I argued that technical moats in software were never that deep to begin with. Looking back on my career in SaaS, the moat was almost never the code. It was off-the-shelf technology organized around a domain someone understood unusually well. So if it wasn’t the code, what were customers paying for?

Mostly, people. They buy the employees with prestigious backgrounds, and the vetting and network that come with them. They buy a company’s pedigree and its story. They buy on faith in named people, in their past performance, their current performance, and their taste. Put simply, we don’t buy products as much as we buy the semi-intangibles behind them, and those are exactly what a company gives away when it lets AI own the value chain.

Taste is the easiest place to see it. I’ve written before that the more we use AI, the more our language converges with it, and taste converges the same way. Every company is renting judgment from the same handful of frontier models. You can steer them with prompts, fine-tuning, and style guides, but the defaults bleed through, which is why so much AI-assisted work already shares a recognizable voice. A company’s taste used to be the accumulated judgment of particular people about what’s good, what’s on-brand, and what should never ship. Rented taste isn’t taste in any competitive sense. It’s a baseline everyone shares. If everything is determined by model taste, everything starts to look like AI slop, and one company’s slop is indistinguishable from another’s.

The problem is that rented taste is also frozen taste. A human organization fixes bad judgment by changing who’s in the room. It hires someone with a different background, a contrarian streak, or a better eye. With AI, you’re limited to prompt engineering and a few frontier models. There’s nobody to fire for bad execution or bad taste, and nobody to hire to bring new ideas until a new model version comes out, on the vendor’s schedule rather than yours. Switching vendors just trades one homogenized taste for another.

Then there’s cover. “Nobody ever got fired for buying IBM” was a running line for decades because it was true. Executives weren’t buying the best mainframe. They were buying a defensible decision. Consulting firms run on the same economics today, charging a premium for recommendations a smaller firm could just as easily make, because the name carries weight in the boardroom and protects whoever signed the contract if the decision goes wrong. “The model recommended it” doesn’t protect anyone.

Accountability is where this stops being theoretical. In 2024, Air Canada’s support chatbot told a grieving customer he could claim a bereavement discount after he’d already booked his flight. That policy didn’t exist. When he took it to a tribunal, Air Canada argued, more or less, that the chatbot was responsible for its own actions. The tribunal rejected that, and Air Canada paid. The company tried to hand accountability to the software, and the law handed it right back. You can’t fire a chatbot, and you can’t sue one. Somebody human is always on the hook, whether they realize it or not.

The loss nobody prices in is the pipeline. Judgment isn’t learned by reading about judgment. It’s formed by producing work, having it torn apart in review, and figuring out why. That loop is how juniors become the senior people who can tell good work from bad. In an earlier piece on vibe coding, I argued that LLMs amplify engineering knowledge rather than replace it, and that if you don’t know what you’re doing, the LLM won’t either. A company that lets AI own the chain stops running the loop that produces people who know what they’re doing. It keeps drawing down the judgment of its current senior staff while training no successors, and in ten years there’s no one left who can tell whether the AI is right.

The fair counterargument is that a company can fine-tune on its own data, or put a strong operator in charge of steering the AI. That works, and in practice it probably means fewer people. But notice what it actually is: humans owning the chain, with AI as leverage. The differentiation lives in data people produced and in the judgment of whoever is steering. That isn’t a rebuttal to the argument. It’s the argument.

Nuance Is Not AI’s Friend

The hardest problem for AI is not artifact generation. It’s nuance.

This is where a lot of AI discourse becomes too shallow. People treat language as the equalizer. If an AI can use language, reason in language, respond in language, and produce convincing language, then the assumption is that it can participate in human decision-making on basically equal terms. But language is not the whole of human intelligence. Language is downstream of human experience.

Humans operate language from embodied experience. We choose words from inside a living system: body, memory, emotion, family history, class background, education, culture, fear, trust, fatigue, shame, ambition, professional incentives, social pressure, and consequence. We don’t just pick the next statistically appropriate sentence.

We ask, often without consciously asking: Who am I talking to? What do they really mean? What are they afraid to say? What are they trying to hide? What will happen if I’m too direct? What will happen if I’m not direct enough? Can this person be trusted? Is this a real objection or a status performance? Is the room actually aligned? Is the customer confused? Is the executive posturing? Is the engineer hiding uncertainty? Is the requirement real, or is it a symptom of a deeper problem?

That isn’t merely language. That’s human-state estimation.

An LLM doesn’t operate from that same state. It operates from abstractions produced by embodied experience. It’s trained on the documentary exhaust of human life: books, articles, emails, transcripts, code, reviews, forum posts, documentation, labels, messages, and feedback. But those artifacts are already abstractions. They’re things humans wrote after living, sensing, filtering, simplifying, exaggerating, rationalizing, omitting, performing, or justifying experience.

Then the model compresses those abstractions again into statistical structure. Then, at inference time, it responds to a partial prompt through that compressed structure. That isn’t one layer of loss. It’s stacked loss. Human experience becomes language. Language becomes training data. Training data becomes tokenized statistical structure. That structure is then sampled from a limited context window to produce more language.

The output can be useful. It can be impressive. It can even be astonishing. But it’s still several abstractions removed from the lived context that gives language its meaning.

This isn’t just a philosophical distinction. It’s a structural one. Language is a lossy projection of human experience, not a sufficient representation of the human state that produced it. Many different internal states can produce the same sentence. “Sounds good” can mean agreement, resignation, fear, politeness, boredom, distrust, strategic compliance, quiet objection, or “I’m documenting that I warned you.” Same words. Different human state. Different correct response.

Knowing what language to use isn’t solved by language modeling alone, because the right language depends on the underlying human context, and much of that context is only partially observable. AI can model the projection. It can learn patterns in what humans say, write, approve, reject, and reward. But it doesn’t naturally inhabit the state-space that produces those patterns.

AI is not coming to the water cooler to gossip anytime soon. And that isn’t a trivial limitation, because gossip isn’t just gossip. In organizations, informal conversation is one of the ways humans transmit hidden state. It’s where people reveal what they actually think, who they trust, what they’re afraid to say in the meeting, which project is politically dead, which customer is annoyed, and which deadline is fake.

Most of that never becomes clean documentation. It may never become documentation at all. It exists as tone, timing, implication, side-channel conversation, reputation, memory, and social consequence. AI can summarize the meeting notes. It can read the Slack thread. It can draft the follow-up memo. But the real organizational context often lives in the human layer around those artifacts. The water cooler isn’t separate from the work. It’s part of how the work becomes intelligible.

AI Is Bounded by Its Senses

AI is also bounded by its senses. Every intelligence is constrained by what it can perceive.

Last time, I mentioned Google DeepMind researcher Tom Zahavy’s argument in Position: LLMs Can’t Jump: abduction, the creative leap behind genuinely novel discovery, depends on sensory experience that LLMs don’t have. The same constraint applies to judgment.

Humans don’t operate from text alone. We operate inside a continuous perceptual field. We sense tone, timing, silence, facial tension, posture, hesitation, mood, emotional temperature, social pressure, and the subtle shift that happens when one sentence lands badly. We also sense ourselves. Our own fatigue, hunger, anxiety, irritation, confidence, fear, and moral discomfort shape what we notice and how we respond.

AI doesn’t naturally inhabit that field. It receives mediated inputs: text, images, audio, logs, APIs, labels, feedback signals, maybe video, maybe sensor data. But those are instrumented inputs. They’re selected, sampled, formatted, and discontinuous. That isn’t the same as living inside a body, inside a social world, with memory, consequence, and risk.

Judgment depends on perception. It depends on noticing what changed in the room, what someone avoided saying, what a customer actually wants, when a stakeholder is being polite but unconvinced, or when the technically correct answer has become socially wrong.

You can give AI more sensors. You can connect it to more tools. You can feed it more context. That will make it more useful. But the constraint remains: intelligence is bounded by observability. If the system can’t perceive the relevant state, it can’t fully reason from that state. It can only infer it from traces. And inference from traces isn’t the same thing as living inside the causal loop that created them.

The Real Value

None of this means AI is fake. AI is obviously real. It’s useful. It’s changing work. It will keep getting better. It will automate tasks, compress workflows, and eliminate roles that are mostly deliverable production wrapped in process theater. But that’s different from saying AI simply replaces professional judgment.

The real issue is simpler: AI exposes the parts of work that were never that deep to begin with. If your job is mostly taking clear instructions and producing a standard artifact, AI is coming directly for that layer. If your value is formatting, boilerplate generation, basic implementation, generic analysis, summarization, or producing plausible first drafts, the market will put pressure on you.

But if your value includes deciding what matters, reading context, understanding people, making tradeoffs, navigating ambiguity, absorbing responsibility, and turning messy reality into useful outcomes, then AI is more likely to become leverage than replacement.

Suleyman may be right that AI will be able to do most of the tasks. That doesn’t mean it should own the work. The companies that get this right will use AI to make their people faster. The ones that hand over the whole chain will find they automated away the part customers were actually paying for.

The future of work isn’t simply “AI replaces workers.” It’s that AI floods the middle layer with output and makes the layers around it matter more: taste, context, judgment, trust, and accountability. Knowing what problem is worth solving matters more. The deliverable was never the whole job. AI just made that impossible to ignore.