One requirement through our AI lifecycle needed 12 manual interventions.
Any process looks good on a slide. The test is to push one real requirement through all of it and count how often a person had to step in. We did that on 22 October 2026, through the lifecycle we redesigned earlier the same month.
Where it stands now
- 12 interventions. 6 were defects in our own factory, fixed during the run.
- 4 more were still open when we wrote the run up. All four were fixed the next day, 23 October 2026 (ODSF-264, ODSF-265, ODSF-266, ODSF-267).
- 2 were human gates doing what they were built to do.
- We have not re-run the pilot. A closed ticket is not a second run, so we cannot yet say the same path now needs fewer than 12.
An earlier version of this page, and of the playbook, said four were still open as at 28 October 2026. They had closed on 23 October 2026; this is the corrected position.
The requirement
“An architecture description can be checked for citations by one factory command.” It was a real gap, found that same day: a pilot for another product had needed a hand-written citation check. Small enough to finish, real enough to matter.
What went through
- Intake: submitted to the live work-record service, which queued an analysis job.
- Analysis: a background analyst produced two stories and chose the brownfield track.
- Approval: an epic and a draft specification, created against a branch working copy.
- Specification: the product owner accepted it. A person, by design.
- Readiness gate: seven mechanical checks, run for both stories before either went into progress. Both passed once the specification was accepted; until then the gate refused them.
- Build: the orchestrator, with a 150-turn budget. One story took 22 turns. The other stopped once, asking for a technical-lead decision it should not make alone, then finished in 17 turns.
- Review, landing and epic close: the technical lead ran the tests and live checks; one command merged it; the epic closed with its retrospective.
The twelve
Read the right-hand column. “Should the process have done it?” separates a defect in our factory from a person doing the job a person is meant to do.
| # | What a person had to do | Should the process have done it? | Where it stands |
|---|---|---|---|
| 1 | Restart the work-record service: it had been running nine-day-old code, so a recent change never took effect | Yes | Fixed 23 October: a factory merge now reloads the long-running processes that run the changed code (ODSF-264) |
| 2 | Re-run the analysis: the daemon kept a 728-character summary, not the 14,774-character answer | Yes | Fixed in the run |
| 3 | Re-route both stories to the right owner before approval | Yes | Fixed in the run |
| 4 | Approve in-process: the approve call takes its epic mode only from configuration | Partly | Fixed 23 October: a product owner can approve one requirement as an epic without changing the project's default (ODSF-265) |
| 5 | Discard the first approval: its stories took a fallback path and lost their routing | Yes | Fixed in the run |
| 6 | Withdraw two tickets filed as the wrong type and refile them as stories | Yes | Fixed in the run |
| 7 | Record the specification's ticket list by hand: the approval's bookkeeping did not land | Yes | Fixed 23 October: approval records its tickets on the spec and in the spec index every time (ODSF-266) |
| 8 | Set aside a stray re-approval whose draft specification collided with this one's id | Yes | Fixed 23 October: two specifications can never be given the same id (ODSF-267) |
| 9 | The product owner accepts the specification | No (a human gate, by design) | Working as designed |
| 10 | Record the decision a story was blocked on | No (a technical-lead decision, by design) | Working as designed |
| 11 | Bind the build working copies before landing | Yes | Fixed in the run |
| 12 | A "help" flag on the id allocator allocated an id | Yes | Fixed in the run |
What it shows, and what it does not
It shows that a requirement can travel intake, analysis, epic, readiness, build, review, landing and epic close with every gate firing where it was designed to. The readiness gate refused both stories until the specification was accepted. The build refused to guess a decision that belonged to the technical lead and stopped to ask.
It does not show the same path without a person watching. The four late fixes are closed tickets, not a second run, so we do not yet know they hold together. Until a repeat run needs fewer interventions, “autonomous” would be the wrong word, and we do not use it.
The thing worth taking away
At least five of the twelve were the same kind of fault: state that one part held and the next part did not see. A stale process, a lost summary, a routing dropped on a fallback path, bookkeeping that did not land, an id clash. Each part worked alone. The first end-to-end run found the joins. If you build a lifecycle from parts, expect your first month to go there.
Across the whole renovation our log of findings stood at 60 as at 28 October 2026. The most common shape is something reporting success for a reason that was not success: a boundary check that found nothing to enforce, a ticket marked done for work that never landed, a build that hit its turn limit and reported ok.
Do this on your own lifecycle
- Pick one real, small requirement and run it through every stage before you roll anything out. A demo with no intervention proves nothing.
- Write down every time a person steps in, and next to each, whether the process should have done it. That split is the whole report.
- Keep the human gates that are there on purpose (acceptance, a design decision) out of the count of defects. They are working.
- Fix the joins first. Look for state one part held and the next did not see: stale processes, lost summaries, ids taken from the wrong place.
- Run it again. A closed ticket is not evidence. A second run that needs fewer interventions is.
What we have not said
This is one requirement, in our own factory, with a person watching. It is a record of what happened, not a benchmark, and it says nothing about how the same path behaves on your system. The ticket ids are in our work record; the table above is the same content as chapter 14 of the full edition.
The AI SDLC Playbook · Our board, with the numbers we would rather not print