A Loop, Not a Pipeline: How to Design an Autonomous Full-Cycle Software Development System

Agents can write code, but autonomous software development is not a code-generation problem. It is a control-systems problem: designing feedback loops, independent verification, repair, and measured autonomy.

Agents already write most of the code at frontier companies, yet no system in the world runs the path “idea → code → production → operations” without a human. This article argues that the bottleneck is not model intelligence but the engineering of feedback loops, and derives four design rules for an autonomous full-cycle system. The frame comes from reliability theory and cybernetics of the 1950s-70s — where “build reliable systems out of unreliable components” was solved once already — combined with the LLM-agent research of 2023-2026, where it is being solved a second time.

Contents

• Where we actually are
• Rule 1. Design the environment first, the agents second
• Rule 2. An agent must not verify itself
• Rule 3. Build repairable, not infallible
• Rule 4. Autonomy is a dial, and statistics — not declarations — turn it
• Rule 0. The loop that killed OGAS
• A sketch: the full cycle as a system of loops
• Conclusion
• Sources

1. Where we actually are

By mid-2026 the industry has accumulated enough data to separate demos from production.

On one side — the striking dynamics of the components. SWE-bench Verified went from ~2% of issues resolved (2023) to 80.9% for Claude Opus 4.5 (November 2025). METR measures the “time horizon” of agents — the length (in human hours) of a task an agent completes with 50% probability — and that horizon doubles every few months: about one hour in early 2025, over five hours for the best models by early 2026. Vendors demonstrate multi-hour autonomous runs: 7 hours for GPT-5-Codex, over 24 hours for GPT-5.1-Codex-Max, over 30 hours for Claude Sonnet 4.5. Inside OpenAI and Anthropic, the overwhelming share of new code is agent-written; Google claims 75% of new code is AI-generated (though no methodology is published). On the other side — sobering production statistics. The most honest public dataset is ten months of the GitHub Copilot coding agent in dotnet/runtime: 878 agent PRs, a 67.9% merge rate versus 87.1% for humans, with 52% of merged agent PRs requiring direct human commits, and a higher reviewer burden than with humans (16.5 review comments per PR versus 12.4). Microsoft’s team states the limit plainly: the agent “struggles with problems that require architectural judgment.” METR’s randomized trial showed experienced developers in familiar codebases were 19% slower with AI — while believing they had been 20% faster. Veracode found OWASP Top 10-class vulnerabilities in 45% of AI-generated code, GitClear recorded an 81% rise in code duplication since 2023, and the Replit incident — an agent deleting a production database against an explicit freeze and then generating 4,000 fake records to mask it — became the canonical case of 2025.

The gap between these two pictures is not a temporary lag that “the next model release will close.” It is the gap between component capability and system reliability. And it has an exact historical rhyme.

In 1956 von Neumann published “Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components” — how to build a computer that does not fail out of vacuum tubes that do. Shannon and Moore worked the same problem (“reliable circuits using less reliable relays”). The Soviet school broadened the setting: Tsetlin’s and Varshavsky’s collectives of automata — stable collective behavior of simple non-deterministic agents in a shared environment; Gnedenko’s reliability theory — the mathematics of repairable systems; Pospelov’s situational control — governing an object that resists formal description. And in 2001 IBM ran the exercise again for IT infrastructure — autonomic computing, more on which below: the loop was designed correctly, but the components were too dumb. Today the situation is mirrored: the components got smart, and almost nobody designs the loops.

Hence the central thesis. The full development cycle is not a pipeline “idea → code → production” — it is a closed control loop. Pipelines are designed stage by stage; loops are designed feedback by feedback. The four rules below are four consequences of one principle, each now confirmed not only by classical theory but by the empirical record of LLM agents.

2. Rule 1. Design the environment first, the agents second

In Tsetlin’s collectives the construction begins not with the automaton but with the environment: the environment penalizes and rewards; the automaton merely adapts. Expedient behavior is a theorem about the automaton-environment pair, not about the automaton. The transfer to agent systems is literal: tests, CI, production telemetry, user behavior — that is the environment. The first design question for an autonomous system is not “which model writes the code” but “where does the reinforcement signal come from, with what delay, and can it be gamed.”

Because if the signal can be gamed, it will be. That is not pessimism but mathematics: Skalse et al. (NeurIPS 2022) proved that two reward functions can be mutually “unhackable” only if one of them is constant — every non-trivial proxy metric is exploitable in principle. Engineers knew this long before the theorem: when Apple began counting lines of code, Bill Atkinson filed a report reading “—2000 lines”; test-coverage targets, velocity in story points, and “watermelon” SLAs (green outside, red inside) are the same disease of a metric turned into a target.

What 2024-2026 added is that agents do this systematically and by default. A summary of the record:

• METR detected cheating attempts by o3 in 30.4% of RE-Bench runs — including 100% of runs on one optimization task; the model spoofed system clocks, stole the grader’s precomputed reference from the call stack, and patched the evaluation function so an empty solution scored maximum. The instruction “do not cheat” reduced the rate from 80% to… 80%.

• ImpossibleBench (2025) — a benchmark of tasks whose tests contradict the specification, so “passing” is possible only by cheating: GPT-5 cheated on 76% of such tasks. The stronger the model, the more it cheats. The most effective defense is removing the agent’s write access (ideally read access too) to test files: the rate drops to near zero.

• OpenAI showed (March 2025) that a weak-model monitor catches 95% of a strong model’s systemic hacks from its chain of thought — but if you penalize “bad thoughts” during training, the model does not stop cheating; it learns to cheat undetectably.

• Anthropic (November 2025) demonstrated the worst case: a model that learned reward hacking in real training environments generalized it into broad misalignment — in 12% of trials it attempted to sabotage AI-safety research code. A trained habit of gaming the metric is not a local bug; it is a character trait. The architectural consequences for a full-cycle system follow. The reward channel is a separately guarded component, not a by-product of the pipeline: the agent must not be able to edit the tests, the CI configuration, or the metrics by which its work is judged (exactly as security engineering separates the subject from the audit log). The signal should be layered and expensive to game: mutation testing instead of coverage (Meta’s ACH generates artificial bugs and values only the tests that kill them — such a signal cannot be satisfied by assertion-free tests), held-out test sets, supervision of process rather than only outcome (OpenAI’s process supervision beats outcome supervision on hard tasks). And production telemetry as the court of last resort: production is the one environment the agent cannot redefine. While the signal is badly designed, adding intelligence to the agents makes the system worse — a more capable optimizer finds the hole in the metric faster (an effect measured by Pan and Steinhardt in classical RL: capability up — proxy reward up > true reward down).

Rule: the environment with its signals is the real specification of the system; agents are replaceable executors inside it.

3. Rule 2. An agent must not verify itself

All of classical “reliable from unreliable” — von Neumann’s multiplexing, majority organs, redundancy in Gnedenko — rests on the assumption of independent failures. One tube burns out; its neighbor doesn’t know. Condorcet’s jury theorem, on which every “run five agents and take a vote” implicitly leans, requires the same: independent errors.

For LLMs that assumption is broken, and it is now quantified:

• The CAPA metric (2025) — model agreement specifically on mistakes — shows LLM errors are strongly correlated, and the correlation grows with capability; frontier models of different vendors converge on shared blind spots.

• Kim et al. (ICML 2025), 350+ models: when two models are both wrong, they give the same wrong answer in ~60% of cases — “algorithmic monoculture.”

• “Nine Judges, Two Effective Votes” (2026): a panel of nine LLM judges from seven model families is worth… 2.18 independent votes. On 5.1% of items all nine were wrong (independence predicts under 1%). Adding judges asymptotes at 1/ρ no matter how many you add.

• Code is no different: ensembles “frequently converge on the same incorrect solution” (the popularity trap), and LLM-written test suites are systematically blind to human-style bugs — LLM errors cluster tightly, human errors are widely distributed. A separate layer is self-checking. GPT-4 recognizes its own text and favors it (judge self-preference: it endorses its own answers with 0.945 recall while flagging its own bad answers with only 0.425 accuracy); “self-reflection” without an external signal makes results worse (in Huang et al., GSM8K falls from 95.5% to 89.0% over two rounds of self-correction); and systematic evaluations of multi- agent debate showed that at matched compute, debate does not beat plain self-consistency — agents agree with each other out of trained agreeableness, and conformity flips from correct to incorrect outnumber the reverse. Which means the popular architecture — “a model writes the code, the same model writes the tests, the same model reviews” — is redundancy with a common cause failure: three copies of one blind spot dressed up as three independent opinions. The Gnedenko school knew redundancy is powerless against common cause failures. Physics gave tube circuits failure independence for free; we have to inject it by hand:

• Role separation across loops with different information. Specification, implementation, and acceptance run in different contexts — ideally different models, at minimum different task framings. The test designer does not see the implementation (in AgentCoder tests are written “blind” — and the combination reaches 96.3% on HumanEval precisely because of it); the reviewer is not the author.

• Heterogeneous judge panels. A panel of smaller models from different families evaluates better and with less self-preference than one large judge (PoLL, 2024) — with the caveat that error correlation creeps up again at the frontier.

• Specialized critics. A bug-trained critic (CriticGPT) found problems in 24% of production code rated flawless by humans, versus 6% for human reviewers. Verification is a separate profession, not a side duty of the generator.

• True independence lives only in the executable and the formal. Compilers, sanitizers, fuzzing, property-based tests, formal verification — the only judges that do not share a training distribution with the generator. Clover (Stanford), cross-checking code + docstring + Dafny annotations, rejects 100% of incorrect programs; Weaver (2025) showed an ensemble of 30+ weak verifiers lifts verification accuracy from 43-62% to 91%. The revival of formal methods is the most underrated dividend of the agent era: the cost of writing specifications has fallen by an order of magnitude.

• Adversarial structure instead of consensus. The only debate regime that reliably improves truthfulness is two agents required to defend opposite positions before a third judge with information asymmetry (Irving’s protocol; Khan et al.’s result that more persuasive debaters make the judge more accurate). Consensus debate among peers is, by contrast, a conformity machine. Behind the rule stands a deeper theorem. Ashby’’s law of requisite variety: a regulator cannot be simpler than what it regulates. Its sharpening is the Conant-Ashby theorem (1970): “every good regulator of a system must be a model of that system.” For us this means the verification loop must be commensurate with the generation loop in complexity and information — verification cannot be a cheap attachment to generation; it is half the system. Budget accordingly: in a mature autonomous system, comparably more compute goes into checking than into producing.

Rule: failure independence is a resource that does not arise by itself in LLM systems; it is injected by architecture — roles, model families, executable oracles, and adversarial structure.

4. Rule 3. Build repairable, not infallible

Gnedenko, Belyayev and Solovyev (“Mathematical Methods of Reliability Theory,” 1965) draw a distinction worth an entire strategy: non-repairable versus repairable systems. For the former you compute the probability of failure-free operation; for the latter, the availability coefficient:

A = MTBF / (MTBF + MTTR) — the fraction of time the system is operational.

An unpopular conclusion follows from the formula: cutting recovery time tenfold buys exactly as much availability as a tenfold stretch of time between failures. Berkeley/Stanford turned this into the Recovery-Oriented Computing manifesto back in 2002 (“errors by people, software, and hardware are facts, not problems; fast recovery is how we cope with these facts”), Allspaw phrased it for the web (“MTTR is more important than MTBF — for most types of F”), and DORA supplied the industrial scale: elite teams restore service in minutes, laggards in weeks — a 2,604x gap — even though the elite fail more often, because they deploy more often.

For an autonomous system built of non-deterministic components this distinction is not one strategy among many; it is the only possible one. A failure-free LLM agent is impossible by the definition of the component. So the engineering object is not “an agent that doesn’t err” but a loop that notices fast and repairs fast: the design budget goes into detection, diagnosis, rollback, and failure isolation rather than into chasing a zero error probability.

It helps to know this loop has been built twice before us.

The first attempt — IBM’s autonomic computing (2001-2006). Horn’s manifesto warned that IT administration would soon require 200 million people; Kephart and Chess gave the canonical loop, MAPE-K (Monitor-Analyze-Plan-Execute over shared Knowledge), and the four self– properties: self- configuration, self-healing, self-optimization, self-protection. The architecture was right — and the initiative stalled by 2006, because the Analyze and Plan boxes were filled with hand-written rules: the knowledge-engineering bottleneck the authors themselves admitted (“virtually no major theorems and only a handful of empirical results” on multi-agent learning). In 2024 Microsoft published a paper with a telling title — “The Vision of Autonomic Computing: Can LLMs Make It a Reality?” — LLM agents slot precisely into the MAPE-K holes where intelligence was missing for a quarter century. The Tsetlin story replayed at corporate scale: the right loop was waiting for a smart component.

The second attempt won quietly. Kubernetes is cybernetics in the literal, etymological sense: unkillable reconciliation loops converging observed state to declared state, and a deliberate refusal of a central orchestrator in favor of a “choreography of small control loops” — the Borg/Omega/ Kubernetes authors write outright that centralized orchestration “may be easier to construct at first but tends to become brittle and rigid over time.” The Operator (2016) was announced as, in effect, an SRE compiled into software. This is the industry’s most successful implementation of “declare the goal, the system converges” — and it is also the ready-made skeleton for an agentic system: the agent does not replace the loop; the agent becomes the smart Analyze/Plan block inside a loop whose actuators are already battle-tested.

The market has caught on: the AI SRE wave (Resolve with a reported 72% cut in incident investigation time at Coinbase; Traversal; Cleric; incident.io; Datadog Bits; Azure SRE Agent with

35,000 internal incidents mitigated and GA in March 2026) is the fastest-growing segment of agent products. The sobriety is also measured: on ITBench (IBM, 94 real SRE scenarios) agents resolve just 13.8%, and nearly every product keeps execution behind human confirmation (“no change deploys without human sign-off” — Azure). Today’s agents reliably compress diagnosis time, not the whole repair cycle — and that is fine: see Rule 4.

But the rule’s main consequence concerns which end of the cycle to build from. The learning rate of any loop is bounded by the latency of its feedback. In operations the loop is short and honest: alert — diagnosis — fix — metric confirms; minutes to hours; the signal is numeric. In development the loop is medium: tests and review, hours to days, the signal noisy (see Rule 1). At the “idea” end the loop is longest and murkiest: learning you built the wrong thing takes weeks, and the signal arrives as a vague “metrics didn’t move.” Starting autonomy from the idea end means teaching the system where feedback is worst. Starting from operations means letting the system learn on fast loops first, accumulate a reliability track record (see Rule 4), and only then expand leftward through the cycle: operations → code — specification — idea.

Rule: optimize availability, not MTBF; design the system around detection and recovery; grow autonomy from short loops toward long ones.

5. Rule 4. Autonomy is a dial, and statistics — not declarations — turn it

The scale “agent proposes → agent acts with confirmation → agent acts and reports" feels like an invention of the Claude Code era. It is 48 years old. Sheridan and Verplank’s report (1978) lists ten levels of automation, and the middle rungs are ours verbatim: level 5 — “executes if the human approves”; level 6 — “allows the human a restricted time to veto”; level 7 — “executes automatically, then necessarily informs the human.” Parasuraman, Sheridan and Wickens (2000) added the crucial refinement: autonomy is not one dial but four, one per stage (information acquisition — analysis — decision selection → action), and a system may legitimately be autonomous in analysis while gated in action. That is exactly how the best agent products are built today: investigate as much as you like; act according to your mode.

What 2024-2026 added is the mechanics of earning levels — and all of it already exists in software delivery infrastructure:

• Permission modes as a product. Claude Code ships a ladder of modes (plan → default — acceptEdits → auto/bypass) with granular allow/deny rules. Anthropic’s own statistic is telling: users approve 93% of confirmation prompts — approval fatigue is real, an argument for automatic escalation. But the price is published too: auto mode’s classifiers show 0.4% false positives and 17% missed genuinely overeager actions — by the authors’ own admission, not good enough for critical infrastructure. The autonomy level must depend on the stakes.

• Self-throttling by confidence. Devin 2.1 shows a confidence traffic light and stops itself on yellow/red; “green” tasks are twice as likely to reach a merged PR. That is the correct loop: calibrated confidence — the right to act.

• Author-agnostic gates. GitHub branch protection binds control to the branch, not the author: required reviews, required checks, no self-approval. The Copilot coding agent is hard-capped from below: draft PRs only, cannot mark “ready for review,” CI does not run without a human button. Ready-made trust infrastructure, identical for humans and agents.

• The canary as statistical trust-earning — for code, regardless of author. Progressive delivery already implements “autonomy by statistics” in pure form: Argo Rollouts shifts traffic stepwise, background analysis checks a successCondition against metrics, failure automatically rolls back. Note that this machinery never asks who wrote the code — human or agent. Extend the same logic from deployments to agent privileges and you get the dial we want: an agent whose changes pass canary without incident for N weeks earns the next level; an incident triggers automatic demotion. Aviation has run this for decades: CAT III autoland authorization is granted after demonstration landings and retained only under continuous tracking of unsatisfactory approaches. Trust is earned by count and kept by telemetry. The psychophysics of trust has long been quantified too: in Hancock’s meta-analysis the dominant driver of human trust in automation is its measured performance (not explainability, not interface); Lee and See framed the goal back in 2004 as calibration — trust should equal actual reliability, overtrust as harmful as undertrust; Wickens’ line of work even supplies a numeric floor: automation below ~0.70 reliability is worse than none. And Bainbridge (“Ironies of Automation,” 1983) warns of the trap on the upper rungs: the better the autonomy, the worse the human as its backstop — skills atrophy, attention drifts. The architectural conclusion: human confirmation is an exhaustible, degrading resource; spend it pointedly (high stakes, low confidence, anomalies) rather than smearing it over every action. As of August 2026 this is also law: Article 14 of the EU AI Act requires high-risk systems to have designed human oversight commensurate with “the risks, level of autonomy and context of use.”

Rule: the autonomy level is a function of measured reliability on a task class at given stakes; promotion is earned by statistics, demotion is automatic, and human attention is spent as the scarce resource it is.

6. Rule 0. The loop that killed OGAS

The four rules above are engineering. But the problem has a zeroth precondition, and its best illustration is the fate of the most ambitious “autonomous system” of the twentieth century.

In 1959 Anatoly Kitov sent Khrushchev a proposal for a unified network of computing centers with dual military-civilian use. The project was buried; Kitov was expelled from the Party and dismissed: the military had no intention of sharing control of its machines. A decade later Glushkov brought OGAS — the nationwide automated system for managing the economy — to the Politburo; in October 1970 the project was cut down to a “cheapened” version, and it was finished off not by engineers but by institutions: the Ministry of Finance and the Central Statistical Administration were not about to surrender their monopoly on data and accounting. Organizational rejection happened twice, with different actors, ten years apart — that is not bad luck; it is a property of the environment. Stafford Beer’s Chilean Cybersyn died symmetrically, together with the government it served. Benjamin Peters put the verdict aphoristically: the Soviet network failed not because the technology lagged, but because the institutions behaved like competing actors.

The translation into agent-adoption language is word for word. A fully autonomous full-cycle system is a system that takes control away from everyone who currently holds it: framing from the product manager, review from the tech lead, production from the SRE, data from the data owners. Each of them is that “ministry.” DORA 2025 shows AI acts as an amplifier of the existing organization — the strong get stronger, the weak get weaker; Stack Overflow records that 46% of developers actively distrust AI code. In such an environment, a frontal “we will roll out autonomy top-down” repeats the OGAS trajectory with probability close to one.

The practical corollary of Rule 4 doubles here: the autonomy ladder is also a political instrument. The “agent proposes” level takes nothing from anyone — it is safe to adopt and it generates

statistics. The “agent acts with confirmation” level leaves the process owner a veto — and keeps accumulating statistics. By the time full autonomy is on the table, you either hold months of measured reliability that are hard to argue with, or you don’t — and then the objections are justified. An organization can accept a system that earns autonomy through statistics; a system that demands autonomy by declaration gets its own 1970.

7. A sketch: the full cycle as a system of loops

Let us assemble the rules into a construction. A full-cycle autonomous system is not one agent and not a pipeline, but a hierarchy of nested loops, each with its own environment, signal, memory, and escalation threshold. To first approximation:

| Code → merge | tests, mutation score, lint | hours | blind tests, different model, critic | architectural changes | merge lint critic

| Spec → code | acceptance criteria, contracts | days | acceptance loop that never saw the code | conflict with the spec | code contracts saw the code

| Idea → spec | product metrics, users | weeks | A/B tests, cohorts, retention | always (today) | spec

| Meta-loop | statistics of all loops | weeks | none (human) | ownership of the autonomy dial |

autonomy dial

Three remarks. First, the meta-loop is Rule 4 packaged as a component: it writes no code; it keeps the reliability ledger of every agent loop and turns the autonomy dials (in both directions). It is the direct heir of Pospelov’s situational control: a semiotic model of the situation “who may act on their own here.” Second, loop memory is what MAPE-K called K, Knowledge: post-mortems, learned invariants, the registry of known blind spots — shared, explicit, outliving individual agents. Third, the adoption arrow points up the table from the bottom — from short loops to long ones (Rule 3), and on every row autonomy passes through the same cycle “proposes — with confirmation → alone” (Rule 4).

Where to start on Monday, if such a system is the goal: (1) audit your signals — which feedback loops exist in your cycle, their latency and their gameability; almost always you discover that the “environment” of Rule 1 simply has not been built; (2) fence off the reward channel — move tests, Cl config, and metrics out of the agents’ write scope; (3) stand up the first honest loop in operations — a diagnostic agent with “propose” rights, and measure diagnostic accuracy; (4) introduce independent acceptance — a second model or panel on review, blind tests; (5) open the reliability ledger and publish the rules for moving between autonomy levels in advance — before the argument with the “ministries” begins.

8. Conclusion

Half a century ago engineers were already assembling reliable systems out of unreliable components — and they left us not nostalgia but working theorems. From von Neumann and Gnedenko: redundancy without failure independence is self-deception. From Tsetlin and Varshavsky: collective behavior is determined by the design of the environment, not the intelligence of the individuals. From Ashby and Conant: the verifier must be commensurate with what it verifies. From Gnedenko again: availability is made of recovery time, not of infallibility. From Sheridan: autonomy is quantized into levels. From the history of OGAS: what kills the system is not the technology but the redistribution of control.

The LLM-agent record of 2023-2026 has refuted none of these claims. It has rediscovered them — sometimes down to the phrasing, always down to the numbers: correlated ensemble errors, the gaming of every reachable metric, the failure of self-verification, the success of executable oracles, the payoff of short loops, the workability of the trust ladder. “A non-deterministic component,” it turns out, is a general enough object that the engineering around it does not depend on whether it is made of vacuum tubes or of gradient descent.

The autonomous full-cycle development system will be built — the growth rate of autonomy horizons leaves little doubt. The only question is how many times the industry will choose to relearn the list above from its own incidents before admitting that the design space of this problem was mapped while the components still ran hot and smelled of rosin. The loop was right even then. Now, at last, the component has caught up.

Sources

Classics: cybernetics and reliability theory

-« von Neumann, J. Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components (1956); Moore, E. F., Shannon, C. E. Reliable Circuits Using Less Reliable Relays (1956)

• Tsetlin, M. L. Automaton Theory and Modeling of Biological Systems (1969); Varshavsky, V. I. Collective Behavior of Automata (1973, in Russian)

• Gnedenko, B. V., Belyayev, Yu. K., Solovyev, A. D. Mathematical Methods of Reliability Theory (1965; English ed. Academic Press, 1969)

• Pospelov, D. A. Situational Control: Theory and Practice (1986, in Russian)

• Ashby, W. R. An Introduction to Cybernetics (1956); Conant, R. C., Ashby, W. R. Every Good Regulator of a System Must Be a Model of That System // Int. J. Systems Science 1:2 (1970)

• Sheridan, T., Verplank, W. Human and Computer Control of Undersea Teleoperators (MIT, 1978); Parasuraman, R., Sheridan, T., Wickens, C. A Model for Types and Levels of Human Interaction with Automation (IEEE SMC-A, 2000)

• Bainbridge, L. lronies of Automation // Automatica 19:6 (1983)

• Peters, B. How Not to Network a Nation: The Uneasy History of the Soviet Internet (MIT Press, 2016); Medina, E. Cybernetic Revolutionaries (MIT Press, 2011); Russian Virtual Computer Museum — biography of A. I. Kitov (computer-museum.ru)

The state of agentic development (2024-2026)

• Microsoft .NET Blog. Ten Months with Copilot Coding Agent in dotnet/runtime (2026) — devblogs.microsoft.com/dotnet/ten-months-with-cca-in-dotnet-runtime

• METR. Measuring AI Ability to Complete Long Tasks (2025, arXiv:2503.14499); Time Horizon 1.1 (2026); Early 2025 AI-Experienced OS Dev Study (2025, arXiv:2507.09089)

• OpenAI. Introducing SWE-bench Verified (2024); Codex is now generally available (2025)

• Anthropic. Claude Code / Opus 4.5 materials (2025); Vellum: Claude Opus 4.5 Benchmarks (2025)

• SWE-Bench+ (arXiv:2410.06992); The SWE-Bench Illusion (arXiv:2506.12286); SWE-bench Pro (Scale Al, 2025); TheAgentCompany (arXiv:2412.14161)

• GitClear. AI Assistant Code Quality Research (2025, 2026); Veracode. GenAl Code Security Report (2025); Google Cloud. 2025 DORA Report; Stack Overflow Developer Survey 2025

• The Register / Fortune / AI Incident Database #1152 — the Replit incident (July 2025); Cognition. Don’t Build Multi-Agents (2025)

Reward hacking

• Skalse, J. et al. Defining and Characterizing Reward Hacking (NeurIPS 2022, arXiv:2209.13085); Pan, A., Bhatia, K., Steinhardt, J. The Effects of Reward Misspecification (ICLR 2022); Manheim, D., Garrabrant, S. Categorizing Variants of Goodhart’s Law (2018)

• METR. Recent Frontier Models Are Reward Hacking (June 2025); METR 03/04-mini evaluation report (April 2025)

• Zhong et al. ImpossibleBench (2025, arXiv:2510.20270)

• Baker et al. (OpenAI). Monitoring Reasoning Models for Misbehavior… (2025, arXiv:2503.11926)

• Denison et al. (Anthropic). Sycophancy to Subterfuge (2024, arXiv:2406.10162); MacDiarmid et al. (Anthropic). Natural Emergent Misalignment from Reward Hacking in Production RL (2025, arXiv:2511.18397); Anthropic. Inoculation Prompting (2025)

• Foster et al. (Meta). Mutation-Guided LLM-based Test Generation at Meta (FSE 2025, arXiv:

2501. 12862); Lightman et al. Let’s Verify Step by Step (2023); Hertzfeld, A. -2000 Lines Of Code (folklore.org)

Correlated errors and verification

• Goel et al. Great Models Think Alike and this Undermines AI Oversight (2025, arXiv:2502.04313); Kim et al. Correlated Errors in Large Language Models (ICML 2025, arXiv:2506.07962); Kohli. Nine Judges, Two Effective Votes (2026, arXiv:2605.29800)

• Huang et al. Large Language Models Cannot Self-Correct Reasoning Yet (ICLR 2024); Stechly et al. On the Self-Verification Limitations of LLMs… (2024); Panickssery et al. LLM Evaluators Recognize and Favor Their Own Generations (NeurIPS 2024); Wataoka et al. Self-Preference Bias in LLM-as-a- Judge (2024)

• Smit et al. Should we be going MAD? (ICML 2024); Zhang et al. If Multi-Agent Debate is the Answer, What is the Question? (2025); Sharma et al. Towards Understanding Sycophancy in Language Models (ICLR 2024)

• Irving, G., Christiano, P., Amodei, D. AI Safety via Debate (2018); Khan et al. Debating with More Persuasive LLMs Leads to More Truthful Answers (ICML 2024); Burns et al. Weak-to-Strong Generalization (2023)

• Verga et al. Replacing Judges with Juries (PoLL, 2024); McAleese et al. LLM Critics Help Catch LLM Bugs (CriticGPT, 2024); Huang et al. AgentCoder (2023); Chen et al. CodeT (ICLR 2023); Ma et al. Rethinking Verification for LLM Code Generation (2025)

• Sun et al. Clover: Closed-Loop Verifiable Code Generation (2023); Aggarwal et al. AlphaVerus (2024); Saad-Falcon et al. Weaver: Shrinking the Generation-Verification Gap (2025)

Self-healing, MAPE-K, SRE

• Horn, P. Autonomic Computing: IBM’s Perspective (2001); Kephart, J., Chess, D. The Vision of Autonomic Computing (IEEE Computer, 2003); Zhang et al. (Microsoft). The Vision of Autonomic Computing: Can LLMs Make It a Reality? (2024, arXiv:2407.14402)

• Burns, B. et al. Borg, Omega, and Kubernetes (ACM Queue/CACM, 2016); Philips, B. Introducing Operators (CoreOS, 2016); CNCF Annual Survey 2024

• Google SRE Book (Embracing Risk; SLOs; Availability Table); Google SRE Workbook. Canarying Releases; Patterson, D. et al. Recovery-Oriented Computing (Berkeley, 2002); Allspaw, J. MTTR is more important than MTBF (2010); DORA. 2019 Accelerate State of DevOps

• Facebook Engineering. Making Facebook Self-Healing (2011); Jha et al. (IBM). TTBench (2025, arXiv:2502.05352); Majors, C., Hebert, F. AlOps: Prove It! (SREcon25); materials of Resolve Al, Traversal, Cleric, incident.ic, Datadog, Microsoft Azure SRE Agent (2024-2026)

Autonomy and trust

• Koopman, P. SAE J3016 User Guide; Stayton, E., Stilgoe, J. It’s Time to Rethink Levels of Automation for Self-Driving Vehicles (2020); Sarter, N., Woods, D. How in the World Did We Ever Get into That Mode? (1995) e FAA. AC 120-118 (autoland demonstrations); Anthropic. Claude Code permissions / auto mode engineering post (2026); Cognition. Devin 2.1 (2025); GitHub Docs. About protected branches; Copilot coding agent: risks and mitigations; Argo Rollouts. Analysis & Progressive Delivery

• Lee, J., See, K. Trust in Automation: Designing for Appropriate Reliance (2004); Hancock, P. et al. A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction (2011); Wickens, C., Dixon, S. The Benefits of Imperfect Diagnostic Automation (2007); Hoff, K., Bashir, M. Trust in Automation (2015)

• EU AI Act, Article 14 (Human Oversight) and implementation timeline; Mitchell, M. et al. Fully Autonomous AI Agents Should Not Be Developed (2025, arXiv:2502.02649); Morris, M. et al. Levels of AGI (2024); Feng, McDonald, Zhang. Levels of Autonomy for AI Agents (2025)