Practical guides · Practical guide

GxP validation test data: choosing normal, boundary and failure cases

Choose test data against a defined requirement, an independent expected result and a reproducible starting state. Random inputs or an expected result copied from the output can conceal errors. The worked datasets show how to plan normal cases, boundaries, errors and the resets needed between tests.

Table of contents

At a glance

How do you select suitable GxP validation test data?

Derive test datasets from current requirements and relevant failure scenarios. Determine expected results independently of the system before execution. Document prerequisites, roles, initial and final states. Select normal, boundary and failure cases for what each demonstrates. This does not imply a universal minimum number of datasets or a requirement to use production data for every test.

  • A dataset is only verifiable when the intended use, prerequisites, and expected adverse effects are defined.
  • Calculate or justify the expectation independently of the tested implementation.
  • Check boundaries and error cases with different statements; modify inputs deliberately first.
  • A reset restores the initial state and must not obscure existing test records.

1. Define the inspection question prior to the dataset

Start with the statement that is to be supported by testing. Does it involve a calculation, correct assignment to the appropriate order, authorization, or handling of invalid inputs? The same numerical value may be relevant for one of these scenarios and meaningless for another. Therefore, the selection should not begin with the first available export.

The regulatory example framework for this article is Human Medicinal Product-GMP. Annex 11 specifies appropriate test methods and scenarios, including parameter limits, data limits, and error handling. Annex 15 addresses predefined criteria and the evaluation of actual results. Its specific OQ examples are presented in the context of documentation and process validation; they are not issued as unchanged test programs for every software system.

The following datasets constitute internal work aids. They present a selection methodology, not a formally prescribed test quantity. Other GxP areas, additional US requirements, and data protection-specific issues require their respective individual evaluations. Technical validity and appropriate data provision are separate considerations that must both be addressed.

Sources: [1] [2] [3]

2. Make input rules and expected process state clearly defined

A boundary case assumes a known boundary. Clarify whether a range includes its endpoints, whether blank values are permitted, and how numeric values are entered. For example, a specification "up to 100" leaves open questions such as whether zero is acceptable or whether decimal values are permitted. Such ambiguities must be resolved technically before a binding should becomes established.

Describe not only the return value. Should a dataset be stored, a follow-up task be generated, or an action be prevented? A system may display an error message while still creating an unintended entry. The selection of appropriate test data should also reveal such relevant side effects. A simple visual comparison of the message may not be sufficient for this purpose.

Distinguish valid technical rules from the currently observed software functionality. The acceptance of a value of 101 does not indicate a breach of the limit, even if a maximum of 100 was originally specified. This represents a deviation between the rule and the observed behavior. Whether the requirement or its implementation must be modified must be justified, not through passive adjustment of the expected value.

Sources: [2] [3]

3. Determine the target independently

Deletion of the expected result from the approved technical rule. A comprehensible manual calculation can suffice for a simple calculation. In the case of more complex transformations, a different set of reference invoices, known technical reference values or a well-founded combination of several proofs may be suitable. It is crucial that not the same untested logic confirms itself.

A second table cell is not automatically independent when it uses the same copied formula with the same error. Similarly, a screenshot of the application does not substitute for a pre-defined expected value. Maintain the professional derivation and its source. An additional person can verify plausibility; however, mere co-signing does not eliminate a shared incorrect foundation.

Define rounding, unit, and tolerance only where professionally justified. In the later example, whole piece counts are exactly multiplied by four; thus, a rounding tolerance is unnecessary. For measured values, the situation might differ. A universal percentage band would be equally inappropriate, and the requirement that every result must always be mathematically exact, irrespective of its intended use, would be unwarranted.

Sources: [2] [3] [5]

4. Maintain a complete dataset trail

A single input file often does not fully describe the complete scenario. Roles, master data, system status, and expected follow-up actions can influence the outcome. The first work aid consolidates these elements. It may be implemented in a test protocol, a guided file, or an appropriate test tool. What matters most is the clear and traceable relationship.

Remove dataset identifier and revision number when data is reused or modified. Reference common assumptions instead of uncontrolled copying into each row. Any deviation from this must be clearly identifiable. A general note "as before" will become difficult to understand after several changes or during an interrupted run.

What to establish alongside the input values
ComponentSpecific contentComment
Assessment question Requirement, relevant failure class and purpose of the case Explains why this specific entry was selected.
Input and Revision All fields required for the case with unique display of empty values Prevents undetected modifications and confusion between empty, zero, and null values.
Prerequisites Role, Master Data, Configuration and Initial State Identical inputs can result in different evaluations under different conditions.
Independently established expected result Result, unit, and expected process effect together with derivation To be determined prior to observation.
Errors and adverse effects Expected rejection and any prohibited write or subsequent action A visible alert alone may not give the full picture.
Reset and Proof of Reset Evidence retained, reset steps and confirmation of the initial state Enables repetition without overwriting previous results.

Sources: [2] [3] [4]

5. Combine typical cases, boundaries, and errors with awareness

First, select a clear and understandable normal case against which calculations and expected workflow can be verified. Add relevant lower and upper limits, as well as values outside the permitted range. Blank entries, incorrect formats, and inconsistent identifiers constitute distinct error categories. A single limit value does not eliminate these differences.

For diagnostically clear cases, initially modify a key condition. If, at the same time, the quantity is invalid and the order is unknown, a rejection does not yet indicate which control has functioned. Subsequently, deliberately combined cases may be meaningful, for example to establish the sequence of tests or to assess interactions. Such selection must be based on a specific question, not a blanket requirement for every possible combination.

The required depth depends on risk, complexity, existing evidence, and remaining uncertainty. ICH Q9(R1) provides an appropriate risk-based framework. It does not specify a fixed number of data sets nor guarantee that a certain number will detect all errors. Therefore, document the covered input classes and remaining limitations rather than merely the total count.

Sources: [2] [5]

6. Define the complete example prior to execution

The following example was not implemented in a product. A planning function processes an order identifier and a pack quantity. The identifier must consist of exactly eight ASCII characters and must correspond to an existing test order. The pack quantity is entered as a whole number from 1 to 100 inclusive, without a sign, leading zeros, or decimal separators. Exactly four containers are to be planned per pack. These limits are example-specific guidelines and do not constitute GMP requirements.

For valid input, expect exactly one planning record with the original order identifier and the calculated container count. In this example, even valid input does not generate a follow-on order. Invalid input should produce a clear rejection, with no planning record or follow-on order. Check identifier length before looking up the order. This distinguishes an invalid length from an unknown order with a valid identifier format.

The common test basis is designated F-1: Only AU000001 is configured as a task; the role of test planner is authorized to perform the function. For each table case, state S0 begins with an empty input form, containing no planning data and no follow-up tasks. The factor of four and the rules remain unchanged. The table values represent the exactly planned inputs; "empty" refers to an actually unpopulated field, not the word "empty".

Sources: [2] [3]

7. Define specific data points and independent expectations

From the example rule, the calculation for container count is equal to package quantity multiplied by four. Thus, 10 packages correspond to exactly 40 containers, 1 package to exactly 4, and 100 packages to exactly 400. These target values were separately recalculated. For invalid inputs, no professional planning outcome is expected; an internally displayed number shall not serve as a substitute for the required rejection without data logging.

The following scenarios apply to F-1 and initiate independently from S0. For each valid input, the identifier AU000001 remains unchanged; exactly one planning dataset is generated with the specified result. "Reject" implies a visible, relevant error, zero new planning datasets, and zero follow-up orders. These jointly defined expectations are part of each row.

The table shows selected classes, not a complete test program. For example, unauthorized roles, extremely large inputs, additional character sets, concurrent usage, and interface redundancies are not tested. Whether such scenarios are required depends on actual usage and risk assessment. The fully described example cases do not claim complete system coverage.

Planned test data, Revision 1: each F-1 and initial state S0
Purpose and ScopeOrder IDPacks per piece as inputIndependently established expected result
T-01 Normal case AU000001 10 Accept; 10 × 4 = 40 containers.
T-02 Lower Limit AU000001 1 Accept; 1 × 4 = 4 containers.
T-03 Upper Limit AU000001 100 Accept; 100 × 4 = 400 containers.
T-04 Below the area AU000001 0 Reject: Quantity outside the range of 1–100.
T-05 Above the area AU000001 101 Reject: Quantity outside the range of 1–100.
T-06 Required value is missing AU000001 leer Reject: Quantity missing.
T-07 Decimal Format AU000001 1,5 Reject the input because it is not a permitted integer.
T-08 Identifier too short AU00001 10 Reject: The identifier has seven characters instead of eight.
T-09 Identifier for long duration AU0000001 10 Reject: Identifier has nine instead of eight characters.
T-10 Formal correct, unknown identifier AU000002 10 Reject: eight characters, but order in F-1 not present.

Sources: [2] [3]

8. Describe reset and repetition in detail as well

After each case, initially secure the actual finding with assignment to data set, execution, and system version. This includes, as illustrated, the entered values, the observed feedback, and the number and content of newly generated planning data sets. In the case of rejection, it must also be verified whether an unexpected follow-up order has arisen. Any intended reset must not make such findings invisible.

For this isolated example, reset R-1 consists of retaining the execution evidence, removing the planned and follow-on orders created by the test from the test dataset under controlled conditions, clearing the form and confirming S0 again. Order AU000001, the test planner role and the factor of four stay in place. This reset applies to the test environment. It does not describe deletion of production GMP records.

If the run is interrupted or the status is unclear, do not simply restart the next case with an assumed S0. Determine what is actually present and document the restoration process. When a repeat test is performed, assign the new result separately. This ensures that an initial error and a subsequent successful result remain distinguishable.

Sources: [2] [4]

9. Handle data origin and modifications appropriately

Synthetic data can be particularly useful when you need to deliberately generate a boundary, an invalid format, or a rare combination. Real data excerpts, in turn, can reveal structures that are absent in a simple design. The key factor is suitability for the specific testing question. The origin of a dataset does not automatically render it representative or inappropriate.

This article does not assess specific data protection obligations regarding personal product data. Separate evaluation of the permissible use and required protective measures is necessary. Masking alone does not address whether relationships, formats, or value ranges have been preserved for technical testing. The validity of such data must be re-evaluated following any changes.

Control changes to data sets or setpoints in a controlled manner. If a new technical rule now specifies five instead of four containers per package unit, a different expected inventory level will result. Previous results must not be adjusted by overwriting reference values to appear compliant with the new rule. Document the reason for any change, affected cases, and required new validation evidence. Existing, unchanged cases may remain acceptable provided their underlying conditions are still met.

Sources: [2] [3] [4]

10. Jointly limit selection and effort

To initiate the process, valid requirements, known failure scenarios, a clearly defined test environment, and access to required master data are required. The technical department confirms rules and intended meanings; IT or system owner clarifies the current status, roles, and reset procedures; QA supports the established path for documentation and traceability. A clear initial case group helps align these contributions.

The effort increases with unclear technical guidelines, complex data relationships, difficult-to-reproduce conditions, and time-consuming resets. Random data sets do not resolve these issues. Start with a key validation question, complete the data and required versus actual conditions, and demonstrate reproducibility. Subsequently, strategically expand the evaluation to include previously unaddressed risks.

  • Valid requirement and specific testing question identified.
  • Input classes and selected cases clearly justified.
  • All relevant fields, roles, and prerequisites have been defined.
  • Target value and process effect derived in advance independently.
  • Relevant adverse, error, and off-target effect cases considered.
  • Evidence of batch assignment, reset, and re-evaluation of release described.
  • Version status, data origin, and remaining coverage limits visible.

Sources: [2] [5]

Project experience and collaboration

Prepare test cases using independently established expected results

Your process owners define the expected decision. With the test team, we prepare inputs that show routine behaviour, a boundary value that exercises the decision rule and a deliberately incomplete dataset that tests error handling. We establish expected results before running the application. A small change may allow an addition to a controlled reference dataset. A new data path also needs a defined starting state, roles and restart behaviour. What matters is the question each case answers.

Frequently asked questions about Test data

How many test data sets do we need at minimum?

A universal minimum number cannot be meaningfully specified. What is essential is determining which requirements, input categories, and relevant failure scenarios are covered and which uncertainties remain. Ten highly similar typical cases may be less informative than a few deliberately distinct cases. Document the rationale for the selection and its limitations. The ten case scenarios presented in this article are an example, not a regulatory requirement.

Sources: [2] [5]

Must test data originate from the production system?

Synthetic data can accurately map a test question and create rare fault conditions. Real data can provide additional relevant structures for other questions. Evaluate the professional suitability and clarify the permissible provision separately. This article does not deal with data protection clearance for personal data and does not establish a general obligation to use real data.

Sources: [2] [5]

Why is a normal case alone insufficient?

It demonstrates only the behavior under the selected conditions. Whether the system appropriately handles boundaries and invalid inputs remains frequently unclear. Derive additional cases from relevant error scenarios. This does not mean testing every theoretically conceivable value. The selection should highlight key differences, for example, zero, empty field, and a decimal value as distinct inputs.

Sources: [2]

What does a independently determined setpoint mean?

The expected value is derived from professional expertise and is not unverifiedly taken from the result of the test application. In the example of this article, this involves a separate calculation: units multiplied by four. For complex functions, an appropriate reference method is required. A second implementation is not automatically independent if it relies on the same unvalidated assumption or contains a copied erroneous formula.

Sources: [2] [3]

Do we need to test every combination of all input errors?

Such a general requirement is usually neither targeted nor justified. Start with distinct error classes and supplement them with relevant interactions. Two simultaneously erroneous inputs could be meaningful when you are specifically investigating the inspection sequence. For the isolated effectiveness of a quantity check, a generally valid data set is relatively easier to evaluate. Risk and open questions determine the selection.

Sources: [2] [5]

What do we investigate when the system displays an error message?

Compare the observed reaction with the pre-defined expectation. In the example of this article, this includes the absence of a planning dataset and a follow-up order. Therefore, a suitable alert during concurrent unauthorized data storage would not constitute a success. The side effects to be assessed depend on the actual workflow; a simple screen message often does not fully reflect the complete process state.

Sources: [2] [3]

How do we prevent one test from compromising the next?

Define an initial state and verify it prior to the next run. First, preserve the actual records, then systematically restore the isolated test batch under controlled conditions. Also consider queues or downstream items that may influence the next case. An appearance-identical form alone does not prove that no data from the previous run has carried over.

Sources: [2] [4]

May we change the target value following an unexpected result?

Only a logically justified change to the basis justifies a corresponding update to expectations. A surprising system outcome alone is not sufficient to justify retrospectively adjusting the test. Verify inputs, domain rules, reference calculation, and implementation. Maintain the original finding and the justified change separately, and determine which cases require revalidation.

Sources: [3] [2] [4]

Who is responsible for preparing the data sets and expectations?

The technical requirements and their significance should be confirmed by the process responsible party. The IT or system responsible party shall determine the required configuration, roles, and reset procedures. The controlled documentation pathway, including review and approval, shall follow internal responsibilities. An additional individual is not a substitute for a reliable baseline. A fixed number of signatures is not prescribed for this work aid.

Sources: [2] [3]

What should we prepare prior to the first test data entry deadline?

Provide valid requirements, known failure scenarios, system status, and required master data. Clarify how test states can be generated and reset. Effort is particularly high due to ambiguous rules, complex data relationships, and difficult-to-reproduce states. A fully developed test scenario highlights these challenges early. A generic dataset count or fixed duration does not reliably reflect the actual effort.

Sources: [2] [5]

Sources and review date

Sources opened on 6 October 2026. The regulatory example framework is Human Medicinal Product-GMP; specific OQ requirements for modules will not be generically applied to software. The three intended calculations were independently verified; product testing was not performed. Data protection obligations are outside the scope of this contribution.

  1. European Commission: EudraLex Volume 4, current human medicinal product GMP overview
  2. EU GMP Annex 11: Computerised Systems, January 2011 revision
  3. EU GMP Annex 15: Qualification and Validation, Revision 2015
  4. EU GMP Chapter 4: Documentation, January 2011 revision
  5. ICH Q9(R1): Quality Risk Management, European Step-5 Edition 2023

Sources checked: .

Daniel Herrmann, Daniel Herrmann Consulting

Daniel Herrmann Consulting

Develop compelling test data from an inspection query

Daniel Herrmann Consulting assists you in selecting appropriate data classes, defining independent expectations, and establishing audits on a traceable initial state.

  • Computer system validation →

    Bring your GxP systems into validated operation, from laboratory software, MES and eQMS to ERP. We define the scope, plan risk-based tests using GAMP 5 and hand the evidence to QA for release.

Send a project enquiry Customer testimonials →