The Prototype That Answers One Question

In This Issue

• Why prototypes become overloaded with features

• How to identify the assumption that puts the idea at risk

• How to turn that assumption into one testable question

• How to choose the smallest credible prototype

• How to decide whether to proceed, change direction, or stop

A prototype is a decision instrument

A prototype is often treated as an early version of the product.

The team starts with a reduced feature list. It builds a simplified interface, connects a model, prepares a demonstration, and calls the result a prototype.

This may produce something impressive. It may not produce useful evidence.

The purpose of a prototype is to reduce uncertainty before the team commits more time, money, data, and engineering capacity. Its value comes from the decision it makes possible.

Before building one, the team should be able to complete this sentence:

The blank should contain one consequential unknown.

Can the model reliably extract the required fields from the documents we actually receive?

Will users trust the recommendation enough to act on it?

Can the workflow complete within the latency available at the point of use?

Can a reviewer identify and correct a poor output before it causes harm?

If the team cannot name the unknown, it is not ready to choose the prototype.

Find the assumption carrying the most risk

Every proposed system rests on assumptions.

Some concern user behaviour. Others concern data quality, model capability, integration, cost, latency, security, or operational ownership. They are not equally important.

The riskiest assumption combines two properties:

1. The system depends heavily on it being true.

2. The team has weak evidence that it is true.

Start by listing the assumptions that must hold for the proposed workflow to succeed. Write them as claims rather than features:

• The source documents contain enough consistent information to support extraction.

• The model can distinguish the cases that require escalation.

• Users will provide the context the system needs.

• The existing application can expose the required data safely.

• The workflow can complete within the acceptable cost and response time.

• A human reviewer can detect important failures.

Then evaluate each assumption against two questions:

• If this is false, how much of the proposed solution fails?

• What evidence do we currently have that it is true?

The assumption with the greatest consequence and the weakest evidence should usually be tested first.

This prevents the easiest part of the idea from determining the shape of the prototype.

A polished chat interface may be easy to demonstrate. It tells you little if the real risk is whether the system can retrieve the correct customer record, respect access controls, or distinguish a routine request from one requiring approval.

Convert the assumption into a question

“Test the AI assistant” is not a testable question.

Neither is “see whether the prototype works.”

The question must be narrow enough that the prototype can produce interpretable evidence. It should identify the capability, the conditions, and the result that matters.

A useful form is:

For example:

That question defines several things immediately:

• The inputs must resemble the real requests.

• The prototype needs access to the relevant knowledge.

• The output must be reviewed against an acceptance standard.

• The result depends on the rate and seriousness of required corrections.

It does not require a complete support application, production authentication, a finished user interface, or every planned integration.

The question sets the boundary.

Define the evidence before building

A prototype should not be judged by whether the demonstration feels promising.

Write the decision rule before running the test.

The rule might specify:

• Which cases will be tested

• What counts as a successful output

• Which failures are tolerable

• Which failures invalidate the approach

• Who evaluates the result

• What decision follows each possible outcome

Avoid relying only on an average score. A system can perform well across common cases while failing badly on the cases that matter most.

Separate at least two dimensions:

Typical performance: Does the prototype handle the normal workload well enough to create value?

Critical failure behaviour: What happens when the input is ambiguous, incomplete, adversarial, or outside the system’s authority?

A prototype does not need production-scale evaluation. It does need enough representative evidence to change a decision.

If every outcome will be interpreted as permission to continue, the prototype is theatre.

Build the smallest credible artifact

“Smallest” does not mean technically trivial. It means excluding anything that does not help answer the selected question.

The right artifact depends on the uncertainty.

Fidelity should exist where the uncertainty exists.

If the question concerns user comprehension, the interaction needs credible content and flow. The backend may be simulated.

If the question concerns technical feasibility, the system may need real data access and a real model call. The interface may be little more than a script.

If the question concerns operational review, the prototype must expose the information and controls a reviewer would actually have.

A prototype becomes wasteful when fidelity is added to areas that cannot change the decision.

Keep the prototype separate from the MVP

A prototype and a minimum viable product are not interchangeable.

A prototype resolves uncertainty. An MVP delivers a constrained but real outcome to actual users under defined operating conditions.

The prototype may use temporary code, manual steps, narrow data, or simulated components. That is acceptable when those choices do not compromise the question being tested.

The MVP must survive real use. It needs ownership, monitoring, support, security, and a path for handling failure.

Confusing the two creates opposite problems.

If a prototype is engineered as an MVP, the team spends heavily before it has evidence.

If an MVP is treated as a prototype, temporary assumptions and weak controls quietly enter production.

Record which parts of the prototype are evidence and which parts are scaffolding. That distinction becomes important if the result is successful and the team decides to continue.

Run the test without rescuing it

Teams often help a prototype succeed during a demonstration.

They rewrite an input, choose a better prompt, remove a difficult case, correct a retrieved document, or explain what the interface was supposed to mean.

Those interventions may be useful during development, but they are part of the result.

Record them.

If the system requires expert adjustment for every case, the test has revealed an operating requirement. If the apparent success depends on knowledge that future users will not have, the workflow has not yet passed.

The prototype should be tested under the conditions named in the question. When those conditions are unrealistic, revise the question rather than quietly improving the environment.

End with a decision

A completed prototype should lead to one of four decisions:

Proceed. The evidence reduced the important uncertainty enough to justify the next investment.

Revise. The underlying need remains valid, but the workflow, architecture, model, or operating assumption must change.

Test another unknown. The first risk has been reduced, and a different assumption now limits the decision.

Stop. The result invalidated a condition essential to the proposed solution.

Stopping can be the most valuable outcome. The team has avoided building a larger system around a weak premise.

The wrong conclusion is: “The prototype went well, so now we should add more features.”

The useful conclusion names what was learned, what remains uncertain, and what the evidence now supports.

The Move

Before starting the next prototype, write a one-page test contract.

Do not begin with the feature list.

Begin with the uncertainty that could make the feature list irrelevant.

Worth Reading

The UK Government Service Manual’s guide to the alpha phase argues that teams should build only what is necessary to test their riskiest assumptions, rather than prototype the entire journey. How the alpha phase works

Takeaways

• Rank assumptions by consequence and strength of evidence.

• Give the prototype one central question.

• Define the evidence and decision rule before building.

• Add fidelity only where it affects the result.

• Treat interventions, corrections, and manual work as findings.

• Separate disposable prototype scaffolding from MVP foundations.

• End with an explicit decision, including the option to stop.

If this helped you, leave a comment or your reaction. I’d like to hear where you landed.

INVENEW exists to help tech builders, operators, founders, and leaders turn AI from experiments into working systems.

Sponsored

👨‍✈️ AI is moving fast. Is your security strategy keeping up?

SECURITY DEMO → GRC strategies for securing cloud AI: Join experts from Amazon Web Services (AWS) and SANS Institute for a hands-on security demo exploring how to apply governance, risk, and compliance (GRC) strategies across modern AI environments. See approaches for managing identity and agent access, protecting sensitive data, strengthening detection and response, and addressing risk across the AI lifecycle.

Note: Third-party company and product names belong to their respective owners and are used for identification and illustrative reference only.