Table of contents
At a glance
Why does data provenance matter when assessing AI for GxP?
Draft Annex 22 does not include a separate section on the provenance and quality of training data. Its specific data provisions address representative test data, verified labels and justified preprocessing in section 5, and independence from development, training and model validation in section 6. Information about development data is still needed to identify overlap and assess separation. The data inventory in this article is our working aid, not a template prescribed by the draft.
Scope and Applicability
This article provides a methodological overview. The specific requirements of the Annex-22 Consultation Draft 2025 should be considered as draft guidance. It addresses topics such as test data selection and independence; not every proposed table row constitutes a literal regulatory mandate.
Data origin linked to model purpose and audit assertion
A dataset may be large yet fail to adequately represent its intended use. Therefore, capture origin, generation conditions, selection criteria, and professional relevance. For image data, location, device, lighting conditions, and defect type may be relevant. For process data, timestamp, unit, and operating status may be critical. Selection should be aligned with the actual application.
The Annex 22 Consultation Draft 2025 is not yet in force. The Commission identifies data quality in training data as a key focus area in its summary. The draft text itself does not contain a dedicated section on this; specific data requirements are addressed in the context of test data and its independence (Nos. 5 and 6). According to No. 2.2, appropriate documentation for supplier models must be available and reviewed by the regulated user. The following inventory fields support this evaluation and represent independent methodological recommendations.
Separate data for development, training, and final validation in a traceable manner. Data used indirectly in the selection of a model or its parameters may also influence the independence of subsequent validation conclusions. A renamed copy retains the same substantive linkage. Technical file separation alone does not address this issue.
Create a data provenance record with verifiable fields
Link data packet, source, time period and responsible body. Complete transformations and labels. With manual marking it should be understandable according to which business rule it was carried out and how its correctness was checked. A label is not automatically true, just because it was already in the source system.
Document known limits. If a site or a rare defect type is missing, this constitutes information for the scope of use. A sound overall assessment must not obscure this gap. The decision may require additional data, limited usage, or an alternative testing method.
| Field | Question to investigate | Example |
|---|---|---|
| Origin | Where and under what conditions were the data generated? | Site, Device, and Data Collection Period |
| Selection | Which cases are included or excluded? | Typical cases and critical subgroups |
| Processing | Which transformation alters the input? | Cutting, normalization or unit conversion |
| Label | What constitutes the correct professional response? | Defect class with justified reference decision |
| Role | What was the package used for? | Training or independent audit |
| Version | Which stage belongs to the model test? | Package designated with change reference |
Example: Images of the same object appear in both training and test sets
A team randomly splits image files into training and test sets. Multiple acquisitions of the same physical object end up in both groups. This can lead to an easier test failure than would occur with truly novel objects. The exact impact depends on the model and the images; this scenario illustrates a potential source of bias.
Therefore, capture the origin at an appropriate grouping level. This can be a physical object, batch, session, or another professionally relevant unit. Assess whether related data may influence an independent evaluation. The draft specifically addresses physical objects in the context of test data independence.
Our own control can utilize known group references and a validated split rule. Document which personnel had access to test data and whether these insights were subsequently incorporated into development or selection activities. A high test result remains interpretable only within the context of these established conditions.
| Test case | Action | Expected finding |
|---|---|---|
| Duplicate file | Search for identical copies in both packages | A known dual presence is detected |
| Related object | Review multiple images of the same object | Visible professional dependence |
| Preprocessing | Traceability or normalization to be verified | Entry justification is supported by legitimate use. |
| Label conflict | Compare two differing labeling indications | Technical clarification documented |
| Later use | Evaluate test data following model adjustment | The assessment conclusion will not be issued as independently valid without prior review. |
Consider subgroup and rare case instances separately
A composite metric may mask a weak subgroup. Describe relevant groups based on intended use, for example, location, device, material, or failure type. Assess whether the volume and distribution of data permit a meaningful inference. The draft requires an appropriate statistical foundation; this article does not specify a universal minimum threshold.
Rare critical cases may require targeted additional data or assessments. A purely frequency-proportional selection might include them only sporadically. The selection must take into account professional and scientific implications. Artificially generated data must not be uncritically regarded as equivalent to real-world conditions; the suitability and limitations must be justified.
Report results for each relevant group as appropriate to support the conclusion. Overall high accuracy alone is insufficient if a critical class is frequently overlooked. Acceptance criteria should be established prior to final evaluation and reflect the intended use.
Monitor changes to data and labels
Cleaning data afterwards changes the dataset and may change what the test demonstrates. Document the reason, scope and impact. A mislabelled case can be corrected, but changing it merely to make poor model performance look acceptable confuses the reference standard with the model result.
Version relevant packages and pre-processing. An identical filename does not prove identical content. The specific technical method may vary; what is essential is a traceable link between data state, model state, and test report. Also obtain documented exclusions and their rationale.
When test data is to be used for a subsequent development cycle, its role must be re-evaluated. The previous test is then no longer automatically considered an independent validation of the adjusted model. A suitable new testing approach should be defined instead of indefinitely reusing the same metric.
Data source passed to the company
Operation requires the known operational boundary. New equipment, different lighting, or altered materials may generate inputs outside the previous data space. The origin record helps identify such differences. It is not only a development basis but also serves as the foundation for future change assessments.
Name individuals responsible for data quality, subject-specific labels, and technical processing. A single individual need not assume all roles. Collaboration should ensure that model metrics and subject matter relevance remain aligned. Open boundaries shall be visibly addressed in the decision-making process for usage.
A data list requires relationships to the model and to its intended use
Describe transformations and labels. An image may have been cropped, a data series normalized, or a result manually classified. These steps influence the subsequent evaluation conclusion. Unclear or systematically erroneous labeling cannot be dismissed through a generally good overall result.
Link data snapshots to the corresponding model and test versions. If data is retrospectively adjusted or reused for additional training rounds, the original statement of its role changes. A summary should visibly illustrate such transitions and prevent inadvertent use of test information for development purposes.
Draft: Link data package, model version, and independent test results
The table links origin, processing, labeling, and the role of a data package to model and test stages. A previously cleaned or re-used dataset for training must be treated as a new stage. Known gaps and under-covered subgroups remain explicitly visible.
| data field | Example entry | Acceptance question |
|---|---|---|
| Origin | Location, device, and physical object. | Are the relevant conditions for origin known? |
| Group | Several images of the same object. | Is potential dependence taken into account? |
| Label | Scientifically validated class with reference. | Is the correct answer justified? |
| Processing | Specified cutting and normalization. | Does the transformation align with its intended use? |
| Role | Training or independent audit. | No unnoticed reuse? |
| Status | Package, model, and test report linked. | Which statement is actually true? |
Next steps
- Scope and relevant data variations describe.
- Identify the origin, selection and preparation.
- Control data roles and version references.
- Assess gaps and changes prior to extended use.
Project experience and collaboration
Link the data package to the model version it supports
With your data owners, we establish which dataset was used for each development or testing purpose. We trace identifiers, selection rules, labels and preprocessing to the particular model version. A small inventory may suffice for clearly separated datasets. If images or records have been renamed more than once, we examine their origins and possible overlaps. Unanswered test questions determine which information to request from the supplier. The inventory supports assessment without presenting each field as an Annex 22 requirement.
Frequently asked questions about Data provenance for AI systems
Does a large dataset automatically demonstrate representativeness?
No. For representativeness, coverage and composition of relevant scenarios and subgroups must be considered. Numerous similar normal cases may fail to capture a rare critical variant. A larger dataset provides value only if the additional data meaningfully enhance the test's conclusions. Justify the selection, known gaps, and the limitations of the intended test.
Are synthetic test data generally equivalent?
Must the user also, in the case of a supplier model, understand the data foundation?
Yes, to the extent that information is required for the evaluation of usage, evidence, and limitations. According to point 2.2 of the draft, appropriate documentation should also be available for a supplier model and be reviewed by the regulated user. A model name alone does not establish the data foundation nor the independence of the test data. The specific documentation required will be derived from these review questions.
Does a random data split make the test automatically independent?
No. Related data, identical objects, or previously known test information may influence the conclusion. Assess the appropriate grouping and actual access during development and selection. Technically separated files do not constitute sufficient evidence of content independence. The split rule should align with the data generation process.
Can we correct defective labels after testing?
A scientifically justified correction may be required. Document the reference basis, scope, and impact on the test conclusion. Avoid altering labels solely based on the desired model outcome. Assess whether a new independent evaluation is necessary. The corrected data set must clearly align with the reported result.
How do we handle data from a later-added location?
Compare its conditions with the described scope of use. Other devices, materials, or acceptance conditions may limit the previous conclusion. Evaluate additional data and relevant supporting evidence. An existing overall metric does not automatically validate a new subgroup.
Sources and review date
The listed regulatory sources form the technical framework; Annex 22 is a consultation draft. Data inventory, selection criteria, and examples are standalone reference tools. No universal manufacturer verification checklist is prescribed.
- European Commission: Annex 22 Artificial Intelligence – consultation draft 2025Draft, review status 14.10.2026. Scope, data, human oversight and model validation; no current regulatory update.
- European Commission: EU GMP Annex 11, Computerised Systems (2011)Current version as per EudraLex inventory as of 14.10.2026. Application, risks, operation, and data lifecycle; specific testing proposals are derived independently.
- European Commission: Consultation on Chapter 4, Annex 11 and Annex 22 (2025)Closed consultation 07.07.–07.10.2025; publication of the consultation does not constitute entry into force.
Sources checked: .
Source of data and model validation to be jointly established
We support you in assessing the current situation, clarifying responsibilities, and implementing appropriate evidence within operations. The type of support that is most effective depends on your system, your team, and the specific objective.
- AI system validation →
Bring your AI system into GxP use with traceable evidence and keep it under control in operation. Once the system and intended GxP use are defined, we validate according to risk, addressing data, supplier, tests, release and change control.

