How to test your first AI-built milestone

Test your first AI-built milestone by checking one useful result, an important boundary and an invalid or incomplete input. Compare the actual output with a result you defined before implementation. Keep evidence of the checks that ran and identify anything that remains unverified.

This is a practical starting point, not a complete testing strategy for every system. Shared apps, sensitive data and consequential automated actions need checks appropriate to their risks.

Define completion before asking for code

A milestone such as “create the upload page” describes an interface. A milestone such as “accept a supported file, show its records and explain invalid rows without changing the original” describes behaviour you can check.

Write the expected result first. Include:

• What input the milestone accepts.
• What the user or operator can do.
• What output or saved state should result.
• What it must leave unchanged.
• What happens when the input is invalid.

For a web app, a visible confirmation may be useful evidence, but also check whether the intended record was saved. For an automation, check the resulting action and how it is recorded. For a local script, examine the output and confirm the source was handled as intended.

Use a small sample with a known answer

Create a few dummy records whose expected results you can calculate yourself. Do not begin with a large private customer dataset. Small examples make discrepancies easier to locate and explain.

Here is a fictional milestone for a local task-report script. It reads task dates and produces a report of items due on or before a supplied reference date. The report must not modify the input file.

For a reference date of 10 June 2026:

This is an example test design, not a report of software we have executed. The rule “due on or before” is explicit; if your project means “before today,” the boundary expectation changes.

Use a supplied reference date in tests rather than depending on the day the check runs. Agree on date formats and timezone handling when they affect the outcome.

Cover a boundary and a missing value

Boundaries are places where a small interpretation changes the result: equal to the limit, just below it, just above it, an empty list, or a permission difference between two users.

For the sample script, test a task due exactly on the reference date. Then test a blank date. The builder should not quietly invent a date unless a fallback is part of the agreed requirement.

If your project does specify a fallback, write a separate check for it. For example:

When preferred_date is blank, use created_date. When both are absent, flag the row for review. A malformed preferred date is an error, not automatically the same as a blank value.

That distinction matters. A missing value, a whitespace-only value and a malformed value may need different treatment. State which rule your project actually requires before implementation.

Check what must not happen

Correct output is only part of the milestone. Think about unwanted side effects:

• Did a report overwrite its source file?
• Did preview mode send a real message?
• Could one user read another user's restricted record?
• Did running the task twice create duplicate entries?
• Did an error leave a partially completed action?

Choose checks that match the work. A read-only report can compare the source file before and after, using its bytes or a checksum. A shared app should test access on the server as well as in the interface. An integration may need a duplicate event test if delivery can repeat.

Avoid adding a large test framework to a tiny task just for appearance. A repeatable check with known inputs and clear expected outputs can be a useful first step.

Ask the AI builder for evidence

Claude Code's official verification guidance recommends giving the tool checks it can run and examining the results. You can apply that general practice to your chosen implementation workflow.

Ask for:

1. The exact check or test that ran.
2. The input or fixture used, without secrets.
3. The expected and actual results.
4. A screenshot when the milestone includes a visible interface.
5. Any manual intervention, retries or checks that could not run.

“All tests passed” is incomplete if you cannot tell what was tested. A green check on a date filter does not prove payment delivery, file permissions or production credentials work.

Record the milestone honestly

Use a short verification note:

Milestone:
Environment:
Input or fixture:
Expected result:
Actual result:
Checks that ran:
Retries or manual steps:
Not checked yet:
Next action:

Keep local checks separate from provider-dependent checks. A simulated event can exercise your own processing logic; it does not prove the external service will authenticate and deliver a real event correctly.

If a check fails, explain the difference and correct the bounded issue before expanding the milestone. Preserve the original evidence so you can distinguish a verified fix from a changed expectation.

Use the result to guide the next step

Once the first useful behaviour and its important boundaries are checked, update the project record and choose the next milestone. Add broader testing as the consequences and complexity grow.

For help preparing the context, see how to hand off a project plan and how to define MVP scope.

Start with PenguinKit to organise the foundation and next steps before implementation. A project plan helps define what to build and check; the implementation still needs real testing in its own environment.