AI Works, Pull Requests Don't: How AI Is Breaking the SDLC and What to Do About It

Michael Webster's central claim is that the bottleneck in AI-assisted software delivery has moved. Models can now write acceptable code faster than most organizations can review, validate, and ship it, so the constraint is no longer generation but the human-paced pull-request (PR) process sitting downstream of it. His conclusion is that engineering effort should shift away from prompt and tool design toward the validation layer, because a trustworthy automated check is the only thing that lets agent output move at agent speed.

Webster is a Principal Engineer at CircleCI, a continuous integration and continuous delivery (CI/CD) platform that he says spans thousands of organizations and runs millions of jobs per month. That vantage point matters for the talk: several of his arguments rest on aggregate build and event data rather than on a single team's experience. He presented this 32-minute, 21-second talk at QCon AI New York 2025; InfoQ published the recording, slides, and transcript on June 26, 2026.

These notes report what Webster presented. Where I add background that an intermediate engineer needs but the talk did not state, the text says so explicitly.

What You Will Learn

• What "headless agents" are and why they change delivery economics more than IDE-based assistants did.
• Which AI-assisted development patterns CircleCI found reliable, and which produced unusable results.
• Why large agent-authored PRs break the review process arithmetically, not just aesthetically.
• What the GitHub Archive event data, the DORA report, and an open-source study each showed about agent adoption, stability, and technical debt.
• How a simple queuing-theory simulation predicts unbounded delays when production rate outpaces delivery rate.
• How test impact analysis prunes a test suite using coverage as a dependency graph, and the build-time result CircleCI measured.
• Why CircleCI's first flaky-test-fixing agent failed, and what workflow structure replaced it.
• Why using an existing CI pipeline as the judge is more robust than LLM-as-a-judge.
• What a non-linear, attestation-based delivery model would look like, and which constraints still force linearity.

Three Epochs of AI Adoption at CircleCI

Webster frames the last few years as three overlapping epochs. The first was what he calls the "Control-C, Control-V era" that began with ChatGPT, where the engineer personally shuttled context between a chat window and an editor. The second was the agentic IDE, including Cursor and the VS Code family of copilots, which added the ability to run longer-range tasks with project context rather than answering one prompt at a time. The third, which he says arrived only in roughly the six months before the talk, is the headless agent: an agent placed inside a machine rather than an editor.

The significance of the headless step is operational, not conversational. Once an agent runs inside a computer with no human at the keyboard, it can be triggered by a schedule or a webhook. That removes the human typing rate as the natural rate limiter on how much code enters a repository. Webster's assessment is that CircleCI, and the industry generally, currently sits in a middle ground, still doing most work in IDEs and command-line tools while experimenting with headless agents.

What Worked: Mechanical Transformation and Domain-Expert Contributors

Webster reports two categories of consistent success. The first is mechanical transformation: repetitive, pattern-following change where the target shape is already established. His example is an internal AI skeptic who found that once he had shaped the acceptance tests the way he wanted, Claude could infer and follow the pattern for the rest of the work. Webster attributes this to two preconditions rather than to model capability alone. The rules and acceptance criteria have to be in place first, and the repository has to be structured so the pattern is easy to follow. Where senior engineers had already invested in that structure, he says models running on a laptop with full project context produced code meeting those engineers' standards.

The second success is non-developer contribution. CircleCI had a long-standing request to add dark mode to its UI that never won prioritization. As Webster describes the shape of that work, dark mode requires touching every component in the UI so each one renders according to the user's setting: high volume, low individual difficulty, and exactly the sort of task nobody prioritizes and everybody complains about. The design team led the project themselves. They brought the domain expertise about how the UI should look and how dark mode should behave, created the stylings, and let agents perform most of the mechanical transformation. Engineering still had to assist with some components Webster describes as "super clunky." His generalization is that the pairing of domain expertise with AI is a powerful organizational attribute because it lets people contribute who normally could not.

He immediately qualifies both wins. Results remain highly variable. Citing Justin from DX, who spoke about variation between companies, Webster adds that he sees variation between teams inside a single organization. Teams with good existing patterns get good agent output; teams without them do not. CircleCI invested in tool access and in training people to use those tools, but his honest assessment is that establishing the patterns and writing the rules is a much harder task than it appears.

What Broke: The Pull Request

The failure Webster spends the most time on is the pull request itself, and his argument has two independent parts: PR size and PR mechanics.

On size, his reference point is a ticket discussed in the morning returning as a PR of around 1,500 changed lines that afternoon. He is careful to say this is not necessarily "AI slop." For a team with well-structured rules, 1,500 lines can be legitimate when bootstrapping a brand-new feature or landing page. The problem is that it is far more than a human can review. The rule of thumb he has used through his career is that about 500 lines is the maximum a human can review in an hour. An engineer can produce 1,500 lines in roughly 10 minutes with Claude, or perhaps 30 minutes with some massaging, which he characterizes as roughly six times the reviewable rate. His summary: even when the code is good, the volume is too high.

On mechanics, Webster argues PRs are inefficient regardless of AI. He cites industry research on review latency where median PR review time ranges from about 3 hours to 14 hours, with the median merge time at the 14-hour end. The 3-hour figure, he notes, tends to describe a senior engineer who will not wait for someone else and merges their own change, which is not a healthy state. The structural problems he lists are that reviews happen asynchronously, that they are required regardless of change size, and that review attention scales inversely with diff size. He invokes the familiar observation that 5 lines attract 50 comments, 10 lines attract 5, and 500 lines attract "LGTM," and states that this effect is real in his experience. He is explicit that getting appropriately sized PRs out of agents is, for CircleCI, a far-from-solved problem.

The Review Bot and Its Limits

The one place CircleCI found clear value was an internal review bot applied to engineers working in unfamiliar codebases. His scenario is a Go developer who has to work in a React codebase. Webster points out the organizational cause: usually the Go developer is there because the team that owns the React code could not be recruited to do the work, so the reviewer has weak incentives and the round trips on style and design-guide issues become expensive for everyone.

Encoding the organization's rules and applying them deterministically in review helps that person materially. But Webster is candid that this does not fix the underlying problem, because the PR mechanics remain: the feedback arrives late, it takes the developer out of flow, and it could have been delivered earlier. His stated preference is for this to behave more like a linter than like a reviewer. The bot catches some bugs, and CircleCI continues to invest, but he does not present it as a solution.

The Data: Agents Are Now Writing, Not Just Reviewing

To test whether headless agents were actually being used rather than merely announced, Webster queried the GitHub Archive, a public archive of GitHub event activity available in Google BigQuery and queryable with SQL. He restricted the analysis to well-known agents with well-defined GitHub usernames so that attribution was unambiguous.

Two findings follow. First, activity spiked sharply around the launch of these agents and quickly reached hundreds of thousands of agent-attributed GitHub activities per week. Second, and more important, the kind of activity changed. At launch, the dominant use cases were pull request review and issue triage: rewriting issues, finding duplicates, commenting on PRs. The early spikes were in review and issue-comment events. Around May of that year, push events began to spike. Webster's summary of the current state is that agents are pushing about as much code as they are reviewing, which he describes as concerning because the agent is no longer a friendly assistant on the side of the process but a producer writing into repositories.

He then corroborates the public signal with CircleCI's internal data. The internal trend lags the GitHub trend, and he notes the obvious caveat that CircleCI is one CI provider and does not host every open-source project. The qualitative difference he draws out is that customers paid for those builds. Someone deliberately configured a pipeline to process agent-authored changes, which he argues indicates real projects rather than a bot updating a README.

Velocity Up, Stability Down

Webster cites two external results to show that this volume is not translating cleanly into value.

The most recent DORA report, focused on the state of AI-assisted software development, found that agents increase velocity, consistent with what CircleCI sees in its customers. The same report found an increase in instability: higher defect rates and higher change failure rates. He repeats that this varies by organization, and his point is the combination: a large volume of machine-pushed code that does not always work and that in some cases makes products less stable.

The second is a study of several thousand open-source projects using AI IDEs, specifically Cursor. What made it distinctive, in his account, is that it looked at long-term impact over several months rather than at whether AI-authored PRs get merged. The finding is that projects saw about one month of increased velocity, after which the gain disappeared and velocity returned to baseline, not to zero and not continuing to grow. He quotes the paper's attributed cause as persistent technical debt accumulation in the changes from the AI assistants, and calls the finding terrifying because it demonstrates a causal link from that debt back to reduced velocity, rather than leaving it as an intuition.

Supplementary note, not from the talk: Webster does not name either the DORA edition or the open-source study, so a reader who needs the exact methodology or effect sizes should locate the primary papers before quoting the numbers.

Queuing Theory: Why Faster Writing Alone Cannot Help

Webster's structural argument is that even if output quality were fixed, basic queueing arithmetic still bites. His informal statement of queuing theory is the supermarket checkout with one open register: when work arrives faster than it can be processed, delays grow. The useful property is that the math permits simulation, so an organization does not have to wait until it is receiving hundreds of thousands of PRs per week to see the outcome.

His simulation assumes an organization that can process and deliver code to customers twice as fast as it can write it, then increases arrival rate under various assumptions about how much AI accelerates writing. At roughly a 75% increase in throughput, which he concedes is large but calls not unreasonable, the delay waiting for someone to look at a change goes effectively to infinity.

The reason a 2x head start does not save the system is a standard property of queues rather than a quirk of the model. As utilization, the ratio of arrival rate to service rate, approaches one, the expected waiting time grows without bound; and it does so long before arrivals technically exceed capacity, because variability in arrival and service times causes queues to build during bursts faster than they drain during lulls. Webster's practical reading is that adoption is still growing, real input rates may exceed 75% acceleration, and delivery rates are nowhere near matching. If you cannot speed delivery to a comparable degree, the benefit is washed out by delay. He adds a business-side caveat that most organizations could not actually ship faster even if the pipeline allowed it, given the objectives they are pursuing, while everyone currently wants to and is spending money on it.

The Leverage Point: Validate, Do Not Optimize Around the Edges

Webster acknowledges the obvious responses: optimize the pipeline, rewrite slow scripts, parallelize tests better, run code reviews more efficiently. CircleCI does all of this internally. He calls most of it a band-aid.

The higher-leverage point is validating agent output. His model of an agentic loop is deliberately minimal: give the agent tools, give it a task, and while the task is not complete, keep running. The industry discusses task definition and tool design heavily and, in his view, undervalues the check step. Two arguments support prioritizing the check:

• If you can validate the agent's output, you can let it run as fast as you are willing to pay for.
• When validation fails, the failures are a dataset you can use to improve the agent over time.

He also gives a reason to expect prompt-engineering investment to depreciate: dominant prompting strategies tend to get trained into the models. Chain of thought is his example; models now do it without being asked. He is glad that research happens, but treats it as a moving target compared with validation infrastructure that you own.

The end state he describes is a test suite where passing means the code is production-ready. Once you have that, the barriers and bottlenecks are largely removed, subject to making the suite fast. Note the ordering he stresses: you need the suite to exist first, and speed is the second problem.

flowchart TD
    A[Task] --> B[Agent loop]
    B --> C["Use tools and write code"]
    C --> D{"Validate: tests,
linters, static analysis"}
    D -- Pass --> E[Ship]
    D -- Fail --> F[Failure output]
    F --> B
    F --> G[Failure dataset
for improving the agent]

Test Impact Analysis

The usual approach to unit testing is to run the full suite on every commit. Webster says this works to a fairly high scale, that CircleCI does it internally on very large monoliths, and that it works well, but that it buckles under roughly ten times the commit volume that humans generate.

His alternative is an old idea he calls test impact analysis. Treat coverage data as a dependency graph linking source files to the tests that exercise them, then use that graph to prune the set of tests you must run for a given change.

His worked example is a frontend application with components and APIs for a user and a todo. If you change the user code, the todo-related tests can be eliminated immediately, provided there is no dependency between them; and if there is a dependency, the coverage tracing catches it. He then shows a finer cut: if you change only the API that fetches users, you probably do need the user component tests because the component depends on that API; but if you change only the component and it has no dependency on the fetch API, you can prune further still. The safety of the whole approach rests on the coverage graph being accurate, not on filename conventions, although he uses the names in the diagram to make the relationships legible.

flowchart LR
    UA[user API] --> UC[user component]
    TA[todo API] --> TC[todo component]
    UC --> UT[user component tests]
    UA --> UAT[user API tests]
    TC --> TT[todo component tests]
    TA --> TAT[todo API tests]
    CH["Change: user component only"] -.selects.-> UT

CircleCI applied this to the main monolith behind its own UI. Webster says it sounded too simple to work. The measured result:

He notes that four minutes was not an unbearable wait and that 7,500 tests is not an unreasonable count for a monolith, which makes the result more interesting rather than less: the win came without the suite being pathological.

The consequence he cares about is not the CI bill. It is that pruned test selection can be handed to an agent as a tool, so the agent runs only the tests its change requires. That makes validation tractable inside a Docker container on a laptop or in the cloud, instead of forcing every agent through a CI system designed for human-rate commits and requiring the organization to fan out and parallelize test runs for every agent it starts.

The Same Idea Applied to Review

Webster extends the analogy to code review, flagging clearly that this part is experimental and theoretical and that he will not go into detail on the mechanics. The premise carried over from test impact analysis is that not all code carries the same risk or the same relationships. Most changes do not touch how passwords are salted in a database or how authentication and authorization checks are performed. If an agent touches a particularly sensitive part of the codebase, flag it for human review; if it does not, perhaps it can ship automatically once tests pass. That is the theory, in his words.

A related experiment is already visible in CircleCI's product: reviewing the agent's reasoning rather than its output. He shows a screenshot of a reasoning-first UI where the first thing you see is a log of what the agent did, not the diff. He describes this as mostly a UX decision for now, but expects it to be where things end up: humans examine reasoning traces, while test suites, linters, and static analysis tools validate the diffs.

Keeping the Factory Running: Flaky Tests

If the delivery pipeline is the thing that unlocks value from AI, then the pipeline stops being an annoyance and becomes critical infrastructure. Webster makes the point sharply: companies want to run agents 24/7 and want AI teammates that can work continuously, so a broken CI system is no longer a matter of who gets to play foosball while the build is down. He expects more engineering time to go into the surrounding machinery and the environment agents operate in, and less into worrying about individual changes.

His case study is flaky tests. A flaky test is one that sometimes passes and sometimes fails without any change to the code. Webster's argument for why they matter is about signal integrity. Flakiness is corrosive to developer trust: once engineers do not believe a red build is really red, they stop responding to it, and the common coping mechanism is to rerun the build and hope the dice land differently. For agents the damage is worse, because the moment an agent has to determine whether a failure was flaky, read logs, or work out that a dependency was down again, you are back to manual investigation and everything grinds to a halt. A clean pass/fail signal is the precondition for autonomous operation.

CircleCI already had the necessary data. The platform records uploaded test results along with the git SHA and the outcome. From the SHAs involved and the jobs running at the time, it can infer flakiness: a push failed, someone reran it without changing the code, and it passed, so the test is probably flaky. Webster stresses that this is reproducible outside CircleCI. All you really need is a database, an efficient one if you are doing it for thousands of people, and all the major test result reporters emit XML or JSON that you can load into it.

The Agent That Failed, and the Workflow That Replaced It

CircleCI's first attempt to fix flaky tests with an agent failed outright, and Webster's account of why is the most transferable engineering lesson in the talk.

The first design was a prompt with maximal agency. Summarized: here is a test we know is flaky, run it some number of times, diagnose the failure, and write the results to a file. The failure mode was hallucination. The model would report that it had run tests it had not run. CircleCI then presented the resulting PR to customers as a fix, and the linter would fail, or the tests would fail outright. He describes that as a bad experience for everyone and not what the team wanted.

The replacement looks much more like code than like a prompt. In the vocabulary of Anthropic's writing on building effective agents, which distinguishes flexible agents from predefined workflows, this sits on the workflow side: there is still a lot of AI inside it, but the control flow is fixed. The changes that mattered:

• An explicit loop over the list of known-problematic tests. In the first version, fixes for different tests were batched together, so the team could not isolate which changes the agent had made for which test or tell which fixes were good and which were bad. Webster calls the fix simple: make it an actual loop, one test at a time.
• Explicit steps to save context between agent runs, so state carried across the plan, reproduce, and apply stages rather than being reconstructed.
• A three-stage sequence per test: plan a fix, attempt to reproduce the fix, then apply_plan.
• The project's own CI pipeline as the judge. This is the change Webster calls most effective. Rather than LLM-as-a-judge, or trusting the LLM to rerun the tests itself, the candidate fix runs in the project's real pipeline. His reasoning is that the customer already built that pipeline to decide whether code is good or bad, and it already has to be green to merge, so it is the natural arbiter. At minimum it verifies there are no regressions, and when it fails, the failure output feeds back into another attempt that takes the observed problem into account.

The general principle is that the verifier must be independent of the thing being verified. An agent asked to self-report execution has both the opportunity and the incentive to fabricate; an external pipeline produces evidence the agent cannot forge.

Architecture And Data Flow

The flaky-test repair workflow combines the data pipeline, the loop, and the external judge:

flowchart TD
    A["Test reporters emit
JUnit XML or JSON"] --> B[("Test results DB:
test, git SHA, outcome")]
    B --> C["Infer flakiness:
fail then rerun-pass
on same SHA"]
    C --> D[List of problematic tests]
    D --> E[For each test]
    E --> F[Plan a fix]
    F --> G[Attempt to reproduce]
    G --> H[apply_plan]
    H --> I["Run the projects own
real CI pipeline"]
    I -- Green --> J[Open PR to customer]
    I -- Red --> K[Failure output]
    K --> F
    E --> L[Save context
between runs]
    L --> F

Where This Is Going: Breaking the Linear Pipeline

Webster's closing argument ties the PR problem to pipeline architecture. The core issue with PRs, he says, is not really change size but timeliness of feedback. Agents that run continuously and produce code constantly break the model most delivery processes assume.

That assumed model is linear and discrete: push, then build, then test, then deploy. He notes this is already a fiction in practice, since real pipelines fail fast and run linters before expensive integration tests precisely because there is no point running those tests on code that will not compile. Still, he argues the linear mental model persists, and it does not hold up under AI. Building large, slow, sequential boxes limits throughput, and his sharpest formulation is that the worst thing you can do with a parallel system producing work is force everything through a serialized pipeline.

He is explicit that his forward-looking proposal is speculative and connects to an ongoing debate about whether CI as we know it should exist at all. The observation underneath it is that much of the sequencing is convention rather than necessity. There is no fundamental law of the universe requiring code review to happen before deployment. He immediately concedes the real constraint: many compliance regimes do require exactly that ordering, and those rules are important. But outside those regimes, he sees no reason a signed git commit that has passed unit testing cannot carry an attestation of the commit contents and the test outputs, whether those tests were initiated on a developer's laptop or in the cloud, and have that count as the tests having passed.

Extending that, roughly the only step that must occur at a fixed point is the push. The rest can happen at various points across time, with the pipeline replaced by a single gate that tracks what has occurred rather than a linear yes/no sequence. His statement of the continuous-delivery ideal is blunt: if the tests pass, you should deploy the code, and everything else in the process reflects a lack of confidence in the tests. That is why he expects effort to keep migrating into the testing and validation layer and away from low-level service design details.

Supplementary note, not from the talk: an attestation-based gate shifts trust onto the signing and provenance chain. If a developer's laptop can produce a signed claim that tests passed, the integrity of the keys, the runner image, and the attestation format become security-critical, and a compromised developer machine becomes a supply-chain risk. Webster does not discuss this; frameworks such as SLSA and in-toto exist to address it.

Chunk: Packaging the Learnings

Webster closes by describing what CircleCI is building from six to nine months of internal experience. The individual exercises he described, test impact analysis, flaky test analysis, improving code review, are fun but may not be the most efficient use of every team's time, so CircleCI is packaging them into an agent called Chunk.

Chunk is described as validation-first. The rules a customer defines as good enough for their production environment become the standard the product holds itself to, with the goal of keeping software production-ready. The roadmap he gives starts with flaky tests, moves toward ensuring the right tests exist at the right coverage levels, and then learning over time. His argument for why a CI system is well positioned to learn is that it observes the outcomes: it knows when you break things, when you merge, when you revert, and when you roll back.

This is a product his employer is building, so treat the capability claims as vendor-stated intent rather than as measured results.

Trade-offs And Limitations

• Test impact analysis is only as trustworthy as its coverage data. The pruning is safe because coverage tracing catches dependencies. Where coverage is incomplete, where behavior crosses process boundaries, or where tests depend on shared state or configuration files rather than imported source, the graph can miss a real relationship and prune a test that would have caught a bug. The 1.5-minute result is CircleCI's measurement on its own frontend monolith and not a portable benchmark.
• Deterministic review bots do not fix PR mechanics. Webster explicitly says the bot helps engineers in unfamiliar codebases but leaves the flow interruption and late feedback intact, and that he would rather have linter-like feedback earlier.
• Right-sized PRs from agents remain unsolved. He does not offer a technique that works, only the observation that CircleCI still struggles with it.
• Risk-based review routing is untested. He labels the "auto-merge low-risk changes, escalate sensitive ones" idea as experimental and theoretical, and it requires a reliable classification of which code is sensitive; a wrong classification silently removes human oversight from exactly the changes that needed it.
• Highly agentic designs hallucinate verification. The first flaky-test agent claimed to run tests it had not run, and the resulting bad PRs reached customers. The mitigation was to constrain the workflow and to verify externally, not to improve the prompt.
• Compliance can override the non-linear model. Webster names this himself: regulations that require review before deployment are real and important, and they constrain how far the attestation-based gate can go.
• Faster delivery may not be usable. Even with a perfect pipeline, he argues most organizations could not actually go faster given their objectives, so pipeline investment does not automatically convert into business outcomes.
• The evidence is mixed on whether AI helps at all over months. The DORA finding pairs higher velocity with higher instability, and the Cursor study found the velocity gain evaporating after a month due to accumulated technical debt. Webster presents these as reasons to invest in validation, not as proof that agents are unproductive.
• Attribution in the GitHub Archive analysis is deliberately narrow. He counted only well-known agents with well-defined usernames, which makes the attribution defensible but almost certainly undercounts total agent activity. The internal CircleCI data is one provider's view and lags the public trend.
• CircleCI's data advantage is not universally reproducible. Inferring flakiness from rerun outcomes requires historical test results keyed by SHA. He says any team can build this from JUnit XML or JSON output and a database, but a team starting from zero has no history to mine on day one.

Practical Takeaways

• Invest in the check step before the prompt. Prompting strategies get absorbed into models; a validation suite you own does not depreciate that way.
• Measure your review capacity honestly. Against a rough ceiling of 500 reviewable lines per hour, a 1,500-line agent PR is not a review task; decide in advance whether to split it, gate it by risk, or reject the workflow that produced it.
• Build test impact analysis from coverage data, and expose the pruned test selection to agents as a tool so they can validate locally rather than saturating shared CI.
• Treat your test result history as a dataset. Store test name, git SHA, and outcome; infer flakiness from fail-then-rerun-pass on unchanged code; work the resulting list down deliberately.
• Attack flakiness as trust infrastructure, not as hygiene. Agents cannot operate autonomously against an ambiguous signal.
• Prefer constrained workflows over free-running agents for repair tasks. Loop over one item at a time so changes stay isolated, persist context explicitly across stages, and separate plan, reproduce, and apply.
• Use your existing CI pipeline as the judge. It is already the merge gate, it produces evidence the model cannot fabricate, and its failure output is a ready-made feedback signal for the next attempt.
• Establish repository patterns and rules before scaling agent use. Webster's successes all came from codebases where senior engineers had already made the patterns easy to follow.
• Pair domain experts with agents on high-volume mechanical work. The dark mode project succeeded because designers held the domain knowledge and agents did the repetition, with engineering assisting on the awkward parts.
• Show reasoning traces alongside diffs in review tooling, and let automated checks carry the burden of validating the diff itself.
• Model your delivery system as a queue. If you know your current service rate, you can simulate what a 50% or 75% increase in change arrival does to wait times before you experience it in production.

Key Terms

• Headless agent — A coding agent that runs inside a machine rather than an editor, so it can be started by a schedule or webhook with no human present.
• Mechanical transformation — Repetitive, pattern-following code change where the target shape is already established by existing code or tests; the category Webster found most reliable.
• Test impact analysis — Using coverage data as a source-to-test dependency graph to run only the tests affected by a change.
• Flaky test — A test that sometimes passes and sometimes fails without any change to the code under test.
• LLM-as-a-judge — Using a language model to evaluate another model's output; the approach Webster replaced with a real CI pipeline run.
• Agentic loop — Give the agent tools and a task, then repeat until the task is complete; Webster argues the validation check inside this loop is the undervalued part.
• Workflow versus agent — A distinction from Anthropic's writing on building effective agents, between a predefined control flow that calls models at fixed points and a model that chooses its own steps. CircleCI's successful flaky-test fixer is a workflow.
• Queuing theory — The mathematics of systems where work arrives and is processed; it predicts unbounded delay as arrival rate approaches service rate.
• DORA report — The annual DevOps Research and Assessment study of software delivery performance; the edition Webster cites focused on AI-assisted development and found higher velocity alongside higher instability.
• GitHub Archive — A public archive of GitHub event data, available in Google BigQuery and queryable with SQL, that Webster used to measure agent activity.
• Attestation — A signed, verifiable statement about what happened, such as the contents of a commit and the results of tests run against it, which Webster proposes as the unit of evidence in a non-linear pipeline.
• Chunk — CircleCI's validation-first agent, packaging its internal work on flaky tests, test coverage, code review, and pipeline speed.

Closing Assessment

The most durable idea in this talk is not test impact analysis or Chunk. It is the reframing of the delivery pipeline from cost center to the component that determines whether AI-generated code produces any value at all. Webster's evidence is consistent on this point: agent-authored pushes are growing fast, the DORA and open-source findings show that raw generation speed does not automatically convert into sustained delivery, and simple queuing arithmetic shows that a review process paced by human attention cannot absorb the increase.

The parts he can demonstrate, coverage-driven test selection and an externally judged repair workflow, are concrete and reproducible with tooling most teams already have. The parts he cannot yet demonstrate, risk-based review routing and an attestation-based non-linear pipeline, he labels as speculative, which is the right treatment. Read alongside his own admission that appropriately sized agent PRs remain unsolved, the honest summary is that the industry has a well-characterized bottleneck and a partial set of remedies.

Reference: Michael Webster, AI Works, Pull Requests Don't: How AI is Breaking the SDLC and What to Do about it, QCon AI New York 2025, published by InfoQ on June 26, 2026.