The Autonomy Ladder: AI, Performance Engineering, and the Place of the Human

At 03:12 the platform noticed a p99 drift on the checkout service. By 03:14 it had correlated the drift with connection-pool exhaustion introduced by the previous evening’s deploy, shifted traffic away from the affected canary, rolled it back, and opened an incident with a written narrative: symptom, mechanism, blast radius, remediation, follow-up. At 09:05 an engineer read the narrative over coffee, ticked “approved”, and closed the ticket. Nothing went wrong. That is precisely what should worry us.

This is not science fiction. Every individual capability in that scene — anomaly detection on latency percentiles, cross-signal correlation, automated canary rollback, generated incident narratives — ships today in some product, at some level of maturity. What does not yet exist is the seamless whole, the night where all of it happens without a human awake. But the trajectory is unambiguous, and it raises a question that the performance engineering community has mostly answered with a reflex rather than an argument.

The reflex goes like this: AI will eliminate the drudgery so engineers can focus on what requires judgment. I have written that sentence myself, more than once. It is true today. But it is also the kind of sentence that ages badly, because it quietly assumes that the boundary between “drudgery” and “judgment” is fixed — that there is a stable line the machines will approach but never cross. An honest prospective exercise has to treat that boundary as the moving object it is.

I come to the question with a bias worth declaring up front. Earlier this year I ran eight load-testing tools against four reproducible server pathologies and published the measurements. The tools built on a synchronous request loop under-reported tail latency by orders of magnitude, precisely when the server was struggling — and they did it without a tremor. The summary statistics were internally consistent. The averages were reassuring. The reports were clean. Nothing in the output announced the lie; the output was the lie. That experience is the lens I bring to what follows, and it reframes the subject: the interesting question about AI in this discipline is not what it can do for us, but what our discipline already knows about trusting instruments that report fluently.

So this essay does three things. It takes stock of what AI actually does in performance testing and observability today, stripped of vendor varnish. It proposes a reading grid — an autonomy ladder, transposed from autonomous driving — designed to outlive any particular tool cycle. And it follows the ladder to the question I actually care about: not will AI replace performance engineers, but at each rung the machine climbs, where does the human move — and what must the human still know how to do?

What the machines already do

Tool lists are the fast-decaying isotope of technical writing, so I will keep the inventory short and organize it by the nature of the work being absorbed rather than by product. Three strata, from bottom to top.

Mechanical work — the scaffolding. LLMs now generate competent k6, Gatling, or JMeter scripts from an OpenAPI spec or a natural-language description. Coding assistants produce load-test boilerplate the way they produce any boilerplate: fast, mostly correct, occasionally confidently wrong. Report generation, test-result summarization, ticket writing — the clerical perimeter of the discipline — is effectively solved. This is the layer nobody will miss.

Analytical work — the correlation. This is where the serious products live. ML-driven anomaly detection (Dynatrace’s Davis, Datadog’s Watchdog, and a growing open-source fringe) learns baseline behavior instead of relying on static thresholds, which matters because the regressions that hurt are exactly the ones static thresholds miss: gradual degradation, bimodal latency distributions, slow drift. Automated root-cause analysis correlates metrics, traces, and logs across service dependencies — work that used to consume senior engineers for hours. Predictive capacity planning forecasts resource exhaustion before it happens. None of this is speculative; all of it is shipping.

Interpretive work — the meaning. The newest and thinnest layer: natural-language interfaces over telemetry (“why is checkout slow?” instead of a PromQL incantation), generated incident narratives, AI analysis of test runs that does not just flag an anomaly but proposes a story about it. Today this layer is more demo than dependable. But it is the layer with the steepest improvement curve, because it rides directly on frontier-model progress rather than on domain-specific ML.

A filter is needed here, because the market is saturated with AI-washing — thresholds rebranded as “intelligence”, three-sigma rules sold as machine learning. The practitioner’s test is simple and worth internalizing: ask what the system learned, from what data, and what happens when it is wrong. A genuine ML capability has answers to all three. A marketing capability has answers to none, and usually changes the subject to a chat interface. The test is not merely hygienic, incidentally — regulators have begun applying their own version of it, and fining the failures.¹

The stock-taking yields a clean pattern: AI has eaten the mechanical work, is biting into the analytical work, and is reaching for the interpretive work. The interesting question is not whether this continues — it will — but how to think about it without rewriting the analysis every six months. For that we need a grid.

The autonomy ladder

The automotive industry spent two decades learning to talk precisely about automation, and condensed the lesson into the SAE levels of driving autonomy — L0 to L5.² The grid transposes to performance engineering and observability with almost no forcing, because the underlying question is identical: who is doing the task, who is monitoring the task, and who is the fallback when the task goes wrong?

Here is the transposition:

One asymmetry is worth naming before anything else, because it survives the entire climb. In driving, L5 means no human. Here it cannot. “Performant” is not a state of the world a system can discover on its own — it is a trade-off between latency, cost, freshness, consistency and risk, and a trade-off requires someone whose preferences it encodes. Our top rung is not human-free; it is human-at-the-objective-function. Hold that thought: the last third of this essay is about what that person has to be good at.

Three honest calibrations before going further.

First, the ladder is a vector, not a scalar. SAE describes a single continuous task with one fallback window measured in seconds. Our field has nothing so tidy: it has a dozen heterogeneous activities, each sitting on its own rung simultaneously, with fallback windows measured in minutes or hours. A team is not “at L2”. It is L3 on auto-scaling, L2 on anomaly detection, L1 on workload modeling and L0 on capacity planning — all at once, on the same Tuesday. So the grid is not used to score an organization; it is used to draw its profile. And the profile is where the risk actually lives, because danger concentrates in the gaps between adjacent activities: an L3 remediation loop fed by L1 workload assumptions is a system acting autonomously on a model nobody has revisited in a year. The average tells you nothing. The spread tells you everything.

Second, where we actually are: the bulk of real-world practice sits at L1–L2. Copilots draft; anomaly detectors flag; humans validate. Genuine L3 exists in narrow slices — auto-scaling is a decades-old L3 loop for one specific failure mode; automated canary analysis with auto-rollback is L3 for another. Meanwhile the marketing routinely sells L4. The gap between the two is where budgets go to die, and the ladder’s first practical use is as a measuring stick: when a vendor says “autonomous”, ask which level, in which bounded domain, with what fallback.

Third — and this is the lesson the ladder brings with it, free of charge, from the domains that climbed it first — L3 is the dangerous rung. Aviation learned this the hard way. The industry that invented the modern autopilot also invented a name for the pilots who could no longer fly without it: children of the magenta line, trained to follow the flight director’s magenta path so faithfully that manual recovery skills atrophied.³ The problem with conditional autonomy is structural: it asks the human to remain vigilant and skilled precisely while removing the daily practice that builds vigilance and skill. The car industry reached the same conclusion and acted on it: several manufacturers decided to skip L3 altogether rather than ship the “human as standby fallback” configuration, judging it less safe than either full attention or full autonomy.⁴

Transposed to our field, L3 looks like this: an on-call engineer paged at 03:00 because the remediation loop hit its confidence boundary and escalated. The engineer must now out-diagnose, in minutes, under pressure, an AI that handles the routine majority of incidents — which means the engineer has not hand-diagnosed a routine incident in months. The escalation is, by construction, the residue the machine could not close. We are building the exact configuration that aviation spent thirty years learning to distrust, and we are building it without the aviation industry’s training regimes, certification requirements, or simulator hours.

There is a second edge to that configuration, sharper than deskilling. Madeleine Clare Elish gave it a name: the moral crumple zone.⁵ A car’s crumple zone absorbs the force of an impact to protect what matters; in a highly automated system, the human operator absorbs the responsibility for failures they had little power to prevent. Same structure, same function. Our 03:00 engineer is not merely under-equipped to out-diagnose the machine — they are structurally positioned to become the answer to “who was supposed to catch this?” when the review convenes. That question returns later in this essay carrying more weight than it first appears to.

An objection is due here, and it is the strongest one available against this essay. If aviation is the cautionary tale, aviation is also the counter-example: commercial flying is now, by an enormous margin, the safest it has ever been, and automation is the principal reason. AF447 is memorable precisely because it is rare — and arguing from the memorable case is exactly the bias the later sections of this essay complain about. Taken seriously, the objection says: automate, and stop romanticizing the hands.

I think the objection is right about the outcome and wrong about the mechanism. Aviation did not get its safety record from automation alone. It got it from automation plus the machinery it built around the human: type ratings, recurrent training, mandatory simulator hours, crew resource management, incident reporting with legal protection, and regulators who intervene when skills erode. That last part is not a metaphor — when flight-data analysis showed manual handling errors increasing, the FAA issued a safety alert telling operators to deliberately promote manual flight, then reissued it four years later.⁶ Automation was allowed to climb because someone was maintaining the humans at the same time.

Software has adopted the first half of that formula with enthusiasm and the second half essentially not at all. No type ratings, no recurrent qualification, no simulator hours, no regulator watching for skill decay. So the aviation analogy does not argue against automation. It argues that automation was never the variable — the training regime was, and we skipped it.

Keep that asymmetry in mind. The rest of the essay climbs the ladder; the climb is real, and mostly desirable. But the ladder is not a slide, and its middle rung wobbles.

Climbing the ladder

What does the climb actually look like, activity by activity? The near-term mechanisms are already visible in today’s products and papers; the projection is mostly a matter of following slopes.

Quality gates learn. Today’s CI/CD performance gates are static thresholds — “fail if p99 > 200 ms” — which makes them simultaneously too strict (flaky failures on infrastructure noise) and too lax (blind to gradual degradation that never crosses the line). The next generation learns baselines across releases and flags statistically significant regressions: not “you crossed 200 ms” but “this release is drawn from a different latency distribution than the last twelve”. This is not a forecast: MongoDB walked exactly that path — manual inspection of graphs, then thresholds, then statistical change-point detection, because neither of the first two caught regressions in time — and published both the method and the cost of operating it.⁷ That is L2 becoming L3: the gate stops being a threshold you set and becomes a judgment you delegate.

Load scenarios tune themselves. Workload modeling — the art of making a synthetic load statistically resemble production — is one of the discipline’s oldest hard problems. AI attacks it from both ends: analyzing production traffic to generate representative profiles, and adapting the scenario during the run (ramp patterns, request mix, data cardinality) based on how the system responds. The load test starts to behave less like a script and more like an adversarial probe searching for the system’s weaknesses.

Impact analysis moves to the pull request. Given a diff, predict which performance characteristics are at risk and run targeted tests — instead of the current binary choice between “run the full soak suite” (too slow for every PR) and “run nothing” (how regressions ship). The functional-testing version has been running at scale since 2018 — a selection strategy learned from historical test outcomes, halving infrastructure cost while still reporting over 99.9 % of faulty changes⁸ — and the performance version is the same idea applied to a far noisier signal. Performance validation stops being a phase and dissolves into the development loop itself.

Optimization becomes continuous. JVM flags, connection-pool sizes, cache configurations, instance sizing: today these are periodic tuning exercises, done when pain justifies attention, then left to rot. Closed-loop optimizers turn them into control systems — observe, adjust, verify, repeat. The database world got there first and its numbers are not subtle: an ML tuner drawing on a repository of previous tuning sessions reached within 94 % of an expert DBA’s configuration in under an hour.⁹ “Set and forget” becomes “observe and adapt”, and a whole genre of consulting engagement quietly disappears.

Dashboards become conversations. The dashboard is a spatial answer to an interpretive problem: we could not make the system explain itself, so we built walls of charts and trained humans to read them. Natural-language interfaces invert this. You describe what you need to understand; the system synthesizes the relevant signals into a narrative. The wall of charts survives for peripheral vision and war rooms, but the primary interface of observability shifts from navigation to dialogue.

Step back from the individual slopes and a second-order pattern emerges, and it matters more than any single capability: as the activities climb, the artifacts of the discipline disappear. The load test stops being an event with a calendar slot and becomes a continuous property of the pipeline. The dashboard stops being a place you go and becomes an answer you receive. The test report becomes a generated narrative; the tuning exercise becomes a background control loop; the capacity plan becomes a rolling forecast.

This matters because professions are secretly structured around their artifacts. The performance engineer’s calendar, deliverables, and organizational legitimacy are anchored to those objects — the test campaign, the analysis report, the tuning sprint. When the artifacts dissolve into ambient automation, the anchor drags. Which raises, with some urgency, the question the last chapters of this essay exist to answer: if the objects of the craft disappear, what remains of the craft?

What resists

Before answering, intellectual honesty requires the counterweight. The ladder is not smooth, and four resistances are structural — not gaps that the next model release closes, but properties of the problem itself.

The unprecedented. Machine learning is, definitionally, learning from precedent. AI-driven diagnosis excels at failures that resemble past failures — and performance is the kingdom of the emergent. The behaviors that hurt most are novel interactions between components that each behave correctly: the cache stampede that only manifests above a traffic threshold nobody tested, the GC pause pattern that only appears with a specific object-lifetime distribution, the retry storm born from three well-intentioned timeout policies meeting for the first time. An anomaly detector will flag the symptom of the unprecedented. Explaining its mechanism requires causal reasoning about a system state no training distribution contains. That capability is improving, but the asymmetry is durable: the machine’s competence is densest exactly where incidents are cheapest, and thinnest where they are most expensive.

The realism problem. A workload model is not a statistics problem; it is a business problem wearing a statistics costume. Knowing that production traffic peaks at 40 kreq/s is trivial. Knowing that the January sales event triples the ratio of anonymous-browse to authenticated-checkout traffic, that the marketing team bought a TV spot for Saturday 20:45, that the batch reconciliation job collides with the morning peak on the first business day of each month — this is knowledge about the world, held by humans, mostly undocumented, and it is precisely what makes a load test predictive rather than decorative. AI can interrogate humans for this knowledge and encode it faster than we can. It cannot originate it.

The accountability boundary. A quality gate that blocks a release, a remediation that shifts customer traffic, an optimizer that resizes a production fleet — these are decisions with owners. Post-incident reviews, compliance regimes, and courts all converge on the same question: who decided? “The AI decided” is not an answer any of them accepts. This is not a temporary legal lag to be waited out; it is how responsibility works in organizations. Autonomy can climb only as fast as someone is willing to own its decisions — which is why the realistic L4 deployments will be bounded domains with explicit contracts, not general autonomy. The bound is not technical. It is the perimeter of what a named human is prepared to answer for.

The epistemic trap. This is the resistance I find most under-discussed, and it is the one this blog has circled before. I once described the WAZE syndrome — our propensity to follow decision-making systems without questioning them, until the ability to question atrophies. Human-factors research had named the phenomenon long before I did and has been measuring it for decades — misuse, in Parasuraman and Riley’s taxonomy: over-reliance on automation producing failures of monitoring and decision bias.¹⁰ Worth knowing, because it means the effect is not a suspicion about AI. It is a documented property of humans supervising machines, with thirty years of evidence behind it. Apply it to observability and something genuinely new appears: observability is the discipline whose product is knowledge about systems. When the diagnosis is AI-generated and no human can independently verify it, a plausible diagnosis and a true diagnosis become indistinguishable from the inside. The incident narrative reads beautifully either way. And unverifiable knowledge is not knowledge; it is trust wearing knowledge’s clothes.

Let me make the trap concrete with the most instructive case I know — one this blog has demonstrated empirically. Coordinated omission is a measurement pathology in which load tools under-report tail latency precisely when the system struggles, because the measuring loop coordinates with the thing it measures. Its signature properties: the averages look fine, the dashboards look fine, the summary statistics are internally consistent — and wrong by orders of magnitude at the tail. Now run the thought experiment: feed those numbers to an AI results-analyzer. Detecting the lie requires reasoning about how the numbers were produced — the interaction between the tool’s scheduling model and the server’s pauses — not about the numbers themselves. Nothing in the data announces the bias; the data is the bias. An analyst AI trained to find patterns in results would find patterns in results, and would narrate them fluently.

Garbage in, eloquent garbage out. The fluency is the danger. A human junior analyst who does not understand a dataset produces hesitant, visibly confused analysis — the confusion is a signal. A language model produces confident, well-structured analysis of data it has no grounds to trust. As the interpretive layer improves, the quality of the prose and the quality of the epistemics decouple, and somebody in the loop has to know how the numbers were made. AI raises the stakes of a lesson the tools taught first; it does not change it.

The human trajectory

Now the question the whole essay has been building toward. Trace the human’s position as the machine climbs, and four stations emerge — not job titles, but modes of engagement with the work.

The operator does the work: writes the script, reads the flame graph, correlates the timeout spike with the deploy. This is L0–L1 territory, and it is where every practitioner today learned whatever they know. Worth stating plainly: the operator station is where judgment is manufactured. Nobody develops intuition for latency distributions by reviewing summaries of them.

The supervisor validates what the machine proposes: reviews the AI’s root-cause hypothesis, approves the generated test plan, confirms the rollback. L2–L3 territory — the present, for well-tooled teams, and the near future for everyone. The station is comfortable and productive, but it lives under the shadow of the wobbly rung: supervision consumes judgment without producing it.

The auditor no longer validates individual decisions — there are too many, made too fast. The auditor designs the verification of the system that decides: samples its diagnoses and independently re-derives them, probes its blind spots (would it catch a coordinated-omission-style measurement bias? test it), defines its confidence boundaries and escalation contracts, decides what the bounded domains of L4 autonomy are and what evidence would expand or shrink them. This is L4 territory, and it is a genuinely senior craft: you cannot audit a diagnostic system without being able to out-diagnose it on a sampled subset — which loops back, uncomfortably, to where auditors can possibly come from.

The intent-setter defines what “performant” means. At the top of the ladder, the human contribution concentrates in the objective function: what latency, at what percentile, for which user journey, at what cost, traded off how against freshness or consistency or carbon. And here something elegant happens to an artifact we already have: the SLO stops being a reporting device and becomes the contract language between human intent and machine execution. An autonomous optimization loop is exactly as good as its objective; a mis-specified SLO, pursued by a competent optimizer, is a mis-specified production system delivered with superhuman efficiency. Writing objectives that mean what we intend — knowing that something will optimize them literally — becomes the discipline’s highest-leverage skill. Performance engineering’s endgame looks less like operating systems and more like legislating for them.

Two tensions cut across this trajectory, and they are where I would place the discipline’s real strategic risk.

The first is the deskilling paradox. Every station on the trajectory presupposes competence manufactured at the stations below it: supervisors need operator instincts to know when a proposal smells wrong; auditors need deep diagnostic skill to re-derive the machine’s conclusions. But the economic logic of automation removes the lower stations first — why pay a junior to write load-test scripts an LLM writes better? Bainbridge named this structure in 1983, in a paper every generation of automation engineers rediscovers: the ironies of automation — automating the easy majority of a task leaves humans with the hard residue, while simultaneously destroying the practice that made humans capable of the hard residue.¹¹ Follow it to its limit and you reach what I think of as the last-engineer problem: the final generation trained the manual way audits the machines competently until retirement, and behind them the pipeline that manufactures auditors has been closed for years. No individual decision along the way was wrong. The system-level outcome is a discipline that can no longer verify its own tools. Organizations that take this seriously will do something that looks economically irrational: deliberately route real work through humans who could be automated — not for output, but as training infrastructure. Aviation again: simulator hours are not for flying the plane; they are for keeping the pilot real.

The second tension is the mutation of the relationship itself. Technologies traverse three statuses. First the AI is a tool: I wield it, it extends my reach, the verbs are mine. Then a colleague: it proposes, I dispose; we disagree; I check its work — the supervisor and auditor stations are precisely structured colleague relationships. Finally, if it becomes reliable enough for long enough, it becomes infrastructure: trusted without deliberation, like DNS, like the compiler, like the electrical grid. And here is the sting: the transition to infrastructure is invisible from the inside. No meeting is held, no decision is made; one day the team simply no longer contains anyone who checks. We audit our tools. We argue with our colleagues. Nobody interrogates their infrastructure — that is what being infrastructure means. Which is exactly the moment the WAZE syndrome stops being an individual cognitive failure and becomes a structural property of the discipline: not an engineer who trusts blindly, but a practice in which blind trust is the built-in default and verification has no remaining habitat.

So I want to reframe the question this essay opened with. The debate is usually staged as “keeping the human in the loop”, and staged that way, the human loses — on speed, on cost, on consistency, eventually on quality, rung after rung. The durable question is different: not whether the human stays in the loop, but whether the loop stays legible to humans. Legible to whom, precisely: not to a central planner wanting an overview, but to the specific person who will have to answer for the system’s behaviour — the only definition that survives contact with an incident review. A legible loop can be audited, contested, corrected, re-derived, owned; an illegible one can only be believed. The lasting role of the performance engineer — the one no rung of the ladder abolishes, because every rung presupposes it — is guardian of the loop’s legibility: the person who ensures that between human intent at the top and machine action at the bottom, there is no segment that nobody left alive can explain.

A practitioner’s wager

Prospective essays owe their readers a falsifiable position, so let me end with mine — what I am actually doing about all this, stated as a wager rather than advice.

I delegate the mechanical layer now, completely and without nostalgia. Script scaffolding, boilerplate dashboards, first-draft reports, summary prose: an LLM writes them, I review. No craft identity is worth defending there; the operator station is for manufacturing judgment, not for ceremony.

I invest in what climbs with the ladder rather than what it eats. Three things, concretely. Systems thinking — the causal, mechanistic understanding of how load, queues, pools, caches, and failure interact; every station up to intent-setter runs on it. The statistics of latency — distributions, percentiles, the arithmetic of tails, the ways measurement lies; this is the auditing skill, the one that catches the eloquent garbage. And the epistemology of measurement — the unfashionable habit of asking not “what do the numbers say?” but “how were these numbers made, and what would they look like if they were wrong?” That last question, I would argue, is becoming the single most valuable question in the discipline, precisely because it is the one the fluent interpretive layer never asks about itself.

And I keep flying manual on a schedule. Some incidents I still diagnose by hand before reading the AI’s narrative — then compare. Some load tests I still design from the traffic data up. Not romanticism, and not distrust: calibration. It is the same reason pilots hand-fly approaches the autopilot could handle better — not because the autopilot is bad, but because the day it disengages is not the day to rediscover what your hands knew. And that is not a private eccentricity: it is precisely what the FAA told operators to institutionalize. Nobody is going to issue us that alert, so it has to be self-issued. The comparison habit has a second virtue: it is a running audit of the machine, an ongoing measurement of exactly where on the ladder my tools actually stand, as opposed to where their marketing says they do.

That is the wager. I might be wrong about the pace — the interpretive layer might stall, or the accountability boundary might hold autonomy at L2 for a decade. But the direction seems robust, and the asymmetry of the bet is what decides it: if I over-invest in legibility and the machines plateau, I have lost a little efficiency. If I under-invest and they do not plateau, I become a child of the magenta line — fluent in approving narratives I can no longer verify, custodian of dashboards I no longer need, in a loop I no longer understand.

The ladder is real, and it is worth climbing; most of what it automates deserved automation. But somebody has to stay qualified to take the controls — and staying qualified, it turns out, is not a state. It is a practice.