Stork Delivery
Build an AI test set from real questions
A small, well-chosen set of business questions gives an AI project a more useful benchmark than a handful of impressive demos.
The problem: curated demo questions miss ordinary AI failures
An AI assistant can look excellent when its builder chooses the questions. The harder test is what happens when a busy colleague asks an incomplete question, uses an old product name, or requests information that is not available.
Collect those cases before you decide the application is ready.
A practical approach
Start with the people doing the job
Ask the intended users for questions they actually need answered. Preserve useful variation in wording. A carefully rewritten question can hide the ambiguity the system will face in practice.
For each case, identify the information an acceptable answer needs, the sources that support it, and the behavior expected when the evidence is missing.
Some cases should end with a clarification or an admission that the answer cannot be established. A system that always produces a confident response is not necessarily doing the task well.
Include the boundaries
Add questions that the user is not authorized to answer, questions about superseded documents, and questions whose answer changed after a source update.
Include requests that could lead to an external action if your application supports tools. Judge the proposed action separately from the quality of the explanation.
Use approved test data or appropriately controlled real examples. Building an evaluation collection should not create a new uncontrolled copy of sensitive records.
Define how you will judge results
A simple review sheet can separate factual correctness, source support, completeness, and appropriate handling of uncertainty. For a workflow, also record whether the correct action was proposed or completed.
Use examples to align reviewers. Two people may interpret “helpful” differently, while agreeing that a response omitted a required contract exception.
Keep part of the collection out of the routine tuning loop. Otherwise, improvements can become narrowly fitted to the questions the team keeps revisiting.
Run it when something changes
Recheck the collection after changes to prompts, models, retrieval, permissions, or source processing. A change that improves one category may weaken another.
Review failures as individual cases before reducing everything to a single score. The consequence of a wrong internal suggestion may differ from the consequence of an unauthorized account change.
A useful test set does not prove that an AI system will never fail. It gives the team a repeatable way to notice regressions, compare changes, and decide which limitations still need a human in the workflow.
The result: a repeatable check of readiness and regression
Build an initial review set with real task questions, ambiguous requests, missing evidence, and access restrictions. Write the acceptable behavior for each and have the task owner review the results. Keep a separate set for checking changes after tuning. The output should show which failures block release and which limitations users can safely work around, with an owner for unresolved cases.