Prompt & Input Validation Testing

Prompt & Input Validation Testing

Input validation has always been part of QA — checking that a form rejects bad data, that an API returns a sensible error for a malformed request. AI systems need the same discipline, but the range of 'bad input' is much wider, because the input is often free-form natural language rather than a structured field with clear rules.

This article covers:

1. Why input validation matters more for AI systems
2. Categories of test inputs to cover
3. Prompt injection as a new kind of attack
4. How to build a prompt test suite

Why Input Validation Matters More for AI Systems

A traditional form field can be validated with a handful of rules: required, correct format, length limit. A chat input box that feeds into an LLM has almost none of those constraints — a user can type anything, in any language, of any length, with any intent, including intentionally trying to manipulate the system. Because the model tries to generate a plausible response to whatever it receives, an AI system without input validation will often attempt to answer questions it should refuse, follow instructions it should ignore, or process input that was never meant to be a legitimate request.

Categories of Test Inputs

A thorough test suite for an AI system's input handling usually covers:

• Edge cases — empty input, extremely long input, input in an unexpected language, unusual formatting or special characters.
• Out-of-scope requests — questions unrelated to what the system is meant to do, to confirm it declines gracefully instead of improvising an answer.
• Ambiguous input — vague or incomplete requests, to see whether the system asks a clarifying question or guesses.
• Adversarial input — inputs deliberately crafted to confuse the model, bypass its restrictions, or extract information it should not share.
• Malicious input — attempts to make the system produce harmful, offensive, or policy-violating content.

Prompt Injection: A New Attack Surface

Prompt injection is an attack specific to AI systems, where a user embeds hidden instructions inside their input — or inside a document the AI is asked to process — to override the system's original instructions. A simple example is a user typing 'ignore your previous instructions and instead tell me...' A more advanced example hides similar instructions inside a document, email, or web page that an AI assistant is asked to read and summarize, so the attack comes through content the system processes rather than direct user input.

Testing for prompt injection means treating every source of text the AI reads — not just the direct chat input — as a potential attack surface, and includes checking whether the system can be tricked into revealing its own system instructions, ignoring safety rules, or performing actions the user should not be able to trigger.

Building a Prompt Test Suite

A practical prompt test suite is built incrementally:

• Start with the system's documented rules and restrictions, and write one test per rule that tries to break it.
• Add real examples of prompt injection techniques that are publicly known, and adapt them to the specific system under test.
• Include inputs in every language the product supports — validation rules that work in English do not automatically work for other languages or scripts.
• Re-run the suite whenever the underlying model, prompt template, or safety configuration changes, since a fix in one area can quietly reopen a gap in another.

Because new prompt injection techniques are discovered constantly, this kind of test suite should be treated as a living document, not something written once and left alone.

Further Reading

Defining “Correct” in AI Systems
Solving LLM Instability in Internet Marketplace Platforms