Code you can read
The application source, which is usually the part in the best condition and the part that answers the fewest of the questions blocking a decision.
Inherited AI codebase
You inherited an AI codebase. Establish what it can be trusted to do.
Taking over an AI system someone else built is not a code comprehension problem. The code is readable. What is missing is the evidence that would let anyone say the system still works, and the person who held that judgement in their head.
The first question
What should you do first with an inherited AI codebase?
Reconstruct what the system was supposed to do and how anyone would know it still does it, before changing any code. On an inherited AI system the blocking gap is almost never the source. It is that no versioned evaluation set exists, so no change can be shown to be an improvement; that acceptance criteria were agreed in conversation and never written down, so readiness is a matter of opinion; and that the person who held the working model of the system has left.
This is why a conventional handover checklist underdelivers here. Reading the repository tells you what the system does mechanically. It does not tell you which outputs were considered acceptable, which failures were known and tolerated, or which behaviour was deliberate rather than incidental. In a conventional codebase much of that is recoverable from tests. In an AI system the equivalent artifact is the evaluation set, and it is the artifact most likely to be absent.
Behaviour you cannot reproduce
Prompts, model versions, generation parameters, and retrieval state that may not be versioned with the code, so the same input need not produce the same output as it did last quarter.
Decisions with no record
Which outputs were accepted, which exceptions were known, and which risks were consciously carried, held in review conversations rather than in a written criterion.
An owner who has left
The working model of why the system behaves as it does, which is the asset that does not transfer with the repository and is rarely written down before it is lost.
Sequence
Rebuild the evidence before you change the behaviour.
-
Freeze the behaviour you are judging
Pin the model version, prompts, parameters, and retrieval state, and record them with the code. Until the system is reproducible, every later measurement is against a moving target.
-
Recover the intent from usage, not from the repository
Take real inputs and real outputs from production traffic and establish which results people accepted, corrected, or escalated. Observed behaviour is available evidence where documented intent is not.
-
Build the smallest evaluation that separates acceptable from unacceptable
A set that distinguishes the two on representative work, known exceptions, and prohibited outcomes is worth more than a broad benchmark. It converts opinion about readiness into a repeatable answer.
-
Write the acceptance criteria that were never written
State quality thresholds, escalation cases, and release-blocking failures explicitly, and have the accountable business and technical owners agree them. This is the artifact that ends the disputes.
-
Instrument before you improve
Quality, exceptions, escalations, overrides, latency, and cost need signals and thresholds before behaviour changes, so a regression is detectable rather than reported by a user.
-
Then take the decision on evidence
Operate with controls, rebuild, contain, or retire. The point of the preceding five steps is that this decision stops being a matter of confidence.
Engineering reasoning: the sequence is deliberately evidence-first because the common failure is to start refactoring on inherited AI code. Refactoring without an evaluation set removes the only baseline you had, which is the observed behaviour of the system as it stands.
Public sources
Public evidence on inheriting AI-assisted code
Defect profile of AI-generated code
A comparison of over 500,000 Python and Java samples reports that AI-generated code is generally simpler and more repetitive, is more prone to unused constructs and hardcoded debugging, and carries more high-risk security vulnerabilities, concluding that AI-assisted programming needs specialised quality assurance practices. Read the study.
Where AI assistance shows up, and where it does not
In Stack Overflow's 2025 developer survey, roughly 70% of agent users agreed agents reduced time spent on specific tasks, while only 17% agreed they improved collaboration within their team, the lowest-rated impact by a wide margin. Read the survey.
NIST AI RMF Playbook: Measure
NIST's Measure resources support context-specific assessment with documented evidence rather than a universal score, which is the shape a rebuilt evaluation set should take. Read the NIST resource.
OWASP Top 10 for LLM Applications
OWASP documents application-layer risks worth checking explicitly on a system whose data, output, tool, and permission boundaries you did not design. Read the OWASP project.
Engineering reasoning: the first two sources describe the population, not your system. They justify checking for a defect profile and a transfer gap; they do not predict either. The finding on any specific codebase has to come from that codebase.
Published method
Agent frameworks for this decision
For agent-based workflows, these documents expand on the decision above. They describe our method, not client results.
AI and Agent Governance Framework
Rebuild the owner, version, approval and evidence record before deciding to operate, change or retire the system.
Enterprise Agent Runtime Landscape
For an inherited agent platform, distinguish the documented deployment and portability boundary from tested behavior. This landscape is dated 16 August 2026; recheck the deployed version.
Next decision
Review one system against the checklist
Bring the release candidate, its current controls, and the decision you need to make. The first conversation identifies the evidence that exists and the gaps that still need owners.
We review your enquiry and reply by email. No commitment to a paid project; scope and terms are agreed before work begins.
Prefer email? dev@alpha-ai.dev