How to QA test an AI built app before you launch it

You asked your AI to add sign in with Google. You open the app and the button is there. You tap it and nothing happens. You tell the AI, and it says it added the button. That is the whole problem with AI built apps in one sentence, and QA is how you get past it.

What QA means here

QA stands for quality assurance. In plain words: does each feature actually work, start to finish, for a real user? When you generate code fast, the work moves from writing to checking. If you skip checking, your customers do it for you, and they come back with complaints even though the app "just worked" when you tried it.

Step 1: write the test list first

Before testing, write what should happen. A one page list is enough.

1. List every feature in plain words ("A new user can sign up, confirm, and see the home screen").
2. For each one, write the expected result.
3. Add the edge cases: wrong password, no network, empty state, back button.
4. Mark the five or six flows a new user cannot live without. Test those first.

This list doubles as your definition of done. If you have a product packet, test against that. See why vibe coded apps never launch.

Step 2: test the whole flow, not the screen

A button existing is not a feature. Check the full path:

• Does the action reach the server, and does the server respond?
• Is the result saved, and is it there after you close the app?
• Does the error state say something useful?
• Does it work on a real device, not only a simulator?

Apple's guideline 2.1 says to test on-device for bugs and stability before you submit. If you use TestFlight for this, our TestFlight guide shows how.

Step 3: give testers evidence to produce

A tester who says "it broke" is not useful. Ask for:

• The steps to reproduce.
• What they expected and what happened.
• A screenshot or screen recording.
• The device and OS version.

The same standard applies to agents. A failure with a recording can be fixed. A vague complaint cannot.

Step 4: fix, then rerun the same flows

The biggest QA mistake is fixing something and not re-running the original test. Send each reproducible failure back for a fix, then run the exact same flow again. Then run the neighbouring flows, because fixes break things nearby.

Step 5: keep a log

Keep one document that lists each feature, its result, and the date. This is your dogfooding document. It keeps the work honest and shows you when you are actually done.

How we do it at SupaKid Studios

Tomi runs testing in two parts. The first part is making sure every feature works. We give QA agents their own virtual machines: a cloud computer with a browser, the tools they need and a dedicated test account. We point them at the app we are authorized to test, along with the intended behavior. They click through the product, try to break the flows, and screen-record what happened.

One of the tools we use is Devin, which has its own VM and can run on Mac, Windows and Ubuntu. For an iOS app it can download the build and start testing. It records a video of what broke and what did not, then starts fixing. Tomi sends it a dogfooding packet of every feature that should work, and Devin can work through it recursively for a shift of six to eight hours.

Every failure becomes a ticket handed to another agent. The loop repeats until the end to end features work. The second part is security, covered in is my AI built app secure.

This is the QA step before we submit client apps for review. It is how we got JetQuest Travel's group travel club app approved and live on the App Store, and part of what our iOS app team does.

What goes wrong

• Testing only the happy path.
• Testing in the builder preview instead of the built app.
• Letting the same agent that wrote the code grade its own work.
• Fixing a bug without a rerun.
• Skipping a real device.

A starter checklist

1. Fresh install, then sign up.
2. Sign out, sign in, reset password.
3. Delete the account if your app supports accounts.
4. Turn off Wi-Fi and use the main feature.
5. Rotate the phone, switch apps, and come back.
6. Pay or subscribe in test mode if you charge.
7. Open every link and settings screen.

Want a team to do this for you?

Book a 30-minute call with SupaKid Studios. We will tell you what a QA pass would look like on your app.