AI code review and testing agents: how to catch what your coding agent missed

Generating code is now the easy part. The work has moved to checking it. Coding agents often do half a job: a feature looks done, works in the happy path, and then a customer finds the broken corner. You used the app many times and it "just worked", and still the complaints arrive. The answer is a QA layer made of agents that did not write the code.

Review and testing are two different jobs

Code review reads the change. It looks at a pull request and flags bugs, risky patterns and things that do not match the codebase. Testing runs the product. It clicks through the app and checks that features work end to end. You want both.

Tools now do each of these on their own. Devin lists Devin Review as a way to find bugs and vulnerabilities before they reach production. The Codex CLI has a /review command to review changes and find issues, per its official page. Devin's use case library lists PR review and QA testing among its use cases.

Step 1: Write the checklist before the agents start

Testing agents need something to test against. Write a list of everything that should work: sign up, log in, the main flow, the edge cases you care about, and what the screen should show. Tomi calls this a dogfooding packet, a document of all the features that should work, and uses it as the QA agent's brief.

Step 2: Add an independent reviewer on every pull request

The reviewer must be a different agent from the writer, ideally on a different tool. Set it up so every pull request gets reviewed automatically and nothing merges without it. Keep review comments short and specific: file, problem, suggested fix.

Step 3: Test on a real machine

A testing agent needs a place to run your app. Devin's docs describe how it tests end to end after creating a pull request: it runs the app, interacts with it through the browser, records a video of the whole run and sends you the recording. It starts from a short test plan focused on the single most important flow that proves the feature works. You can watch a recording instead of rerunning everything yourself. For the machine side, see giving agents their own computer.

Step 4: Close the loop

When a test fails, the agent should fix it and run again. Tomi says Devin proactively starts fixing what its testing finds. Set a limit on how many loops it can run before it asks for help.

Step 5: Staging, then a human

Release to staging. A person checks the result and signs off. Review catches code problems, testing catches behavior problems, and the human catches "this is not what I meant".

How we do it at SupaKid Studios

Tomi splits testing into two parts. The first is making sure every feature works. He uses a range of products for this. In his QA and security video he talks about Devin, which has its own cloud machine and can run on Mac, Windows and Ubuntu. If he is building an iOS app, Devin downloads the app, tests it, and records a video of it using the app, showing what broke and what did not. Then it starts to fix things on its own.

He sends Devin off with the packet of everything that should work, and it works through it recursively. By his description, it will do a work shift of six to eight hours.

In his larger factory, the order is: sub agents implement and open a pull request, and Devin does QA on that pull request using its own computer. A dev review bot then reviews the code that merged. Release goes to staging, and Tomi checks staging, verifies QA, and then it rolls out. For the whole shape, read what an AI software factory is.

The second part is security testing, which is separate from feature testing and keeps going after launch. We cover it in our QA and security video and in our guide to securing an AI-built app.

For SkipHold, a white-label AI communications platform for phone and IT providers, we plugged their team into AI development and software factories: agents that build, test and ship code.

What goes wrong

• Testing without a checklist. The agent tests what is easy, not what matters.
• Same model, same blind spots. A reviewer on the identical setup as the writer repeats its mistakes. Vary the tool.
• Trusting a pass. A pass means the checklist passed, not that the product is good. Read the recording.
• Endless fix loops. Cap retries.
• Skipping the human step. Especially for design and creative work, review the actual result yourself.

Get QA that runs without you

A QA layer is the difference between code that was generated and software that ships. SupaKid Studios runs this for clients as part of a managed team. If your agents are shipping things that "mostly work", book a 30-minute call and we will look at where your checks are missing.