Remediation gap report before EU launch

Why a gap report matters before launch

When you prepare to release an AI system in the EU you must show that you have met the obligations set out in the AI Act. A gap report turns those legal requirements into a concrete list of tasks you have completed and tasks that remain. This list is useful for internal teams, auditors, and regulators because it shows exactly where work is needed. It does not promise compliance; it simply shows the current state of effort.

We start by reading the official text of the Act and the related guidelines from the European Commission. Each obligation is copied verbatim into a spreadsheet column. Next to each obligation we add a column for evidence (such as a policy document, a test result, or a code comment). If the evidence column is empty we mark the row as a gap. The resulting table becomes the work list that drives remediation.

Because the Act is written in legal language we translate each sentence into plain actions. For example, “provide clear information about the AI system’s capabilities” becomes “create a user‑facing description that lists input types, output formats, and known limitations”. This translation step prevents misinterpretation and makes it easy to assign owners.

How we map Act obligations to technical controls

Mapping is the core activity that turns legal text into engineering work. We use a three‑column table: Obligation, Control, Evidence. The Obligation column contains the exact wording from the Act. The Control column lists the concrete technical or procedural measure we have put in place to satisfy that wording. The Evidence column holds a reference to an artifact (a document link, a test log, or a configuration file).

Below is an example of the table we maintain. It shows a subset of obligations related to transparency and data governance. The table is kept in a shared spreadsheet so that product, legal, and engineering can update it in real time.

Obligation (Act wording) | Control (what we do) | Evidence (artifact) Providers shall ensure that natural persons are informed that they are interacting with an AI system (Article 13(1)). | Add a persistent banner on the chat interface that reads “You are talking to an AI assistant”. | Screenshot of the banner in staging environment (link: internal/wiki/ai-banner). Providers shall make available, in a clear and accessible manner, information about the capabilities and limitations of the AI system (Article 13(2)). | Publish a model card that lists intended use, known failure modes, and performance metrics. | Model card PDF stored in the compliance repo (path: docs/model-card-v2.pdf). Providers shall implement appropriate data governance measures to ensure training data is relevant, representative, and free of errors (Article 10(1)). | Run a nightly data quality script that checks for missing values, label inconsistencies, and out‑of‑distribution samples. | Script output log for the last run (link: internal/scripts/data-quality.log). Providers shall document the steps taken to mitigate bias in the training data (Article 10(2)). | Maintain a bias‑mitigation log that records each re‑weighting or re‑sampling action taken. | Log file in the bias‑mitigation folder (path: docs/bias-mitigation.log).

The table makes it easy to see which obligations have evidence and which do not. Missing evidence rows become the items in the remediation work list. We review the table at each sprint planning meeting and assign owners to close the gaps.

Building the work list: steps we follow

Our process for turning the Act into a work list consists of five repeatable steps. Each step is performed by a small cross‑functional team that includes a legal analyst, a product manager, and an engineer.

- Collect source texts – download the official Journal version of the Act, the Commission’s FAQ, and any relevant standards (for example the OWASP LLM Top 10 OWASP LLM Top 10 or the NIST AI RMF NIST AI RMF). Store them in a read‑only folder. - Extract obligations – copy each sentence that imposes a duty on providers into a master list. Use verbatim wording; do not paraphrase at this stage. - Translate to actions – for each obligation write a short imperative sentence that describes what must be done to satisfy it. Keep the translation under 20 words where possible. - Match to controls – look at existing policies, code, and documentation to see if any already meet the translated action. If a match exists, link the evidence; if not, flag a gap. - Prioritize gaps – assign a priority based on risk level (high, medium, low) and effort estimate (small, medium, large). High‑risk, low‑effort gaps are tackled first.

We repeat the cycle whenever the Act is amended or when we release a new version of the model. The master list lives in a Confluence page that is editable only by the compliance lead, ensuring a single source of truth.

Example: Transparency obligations (Article 13)

Article 13 focuses on making sure users know they are dealing with an AI system and understand what it can and cannot do. To satisfy this we have taken three concrete actions.

First, we added a visible disclaimer to every user‑facing endpoint. The disclaimer appears before the first interaction and remains until the user dismisses it. It contains the exact phrase required by the Act: “You are interacting with an AI system”. The disclaimer text is stored in the UI strings file (path: src/i18n/en.json) and is checked by our automated UI test suite.

Second, we created a model card that summarizes the model’s intended use, known limitations, and performance numbers. The card follows the format suggested in the OWASP LLM Top 10 OWASP LLM Top 10 guidance for transparency. It is hosted on our internal documentation site and linked from the product’s help page.

Third, we set up a quarterly review where the product team verifies that the disclaimer still appears after each UI release and that the model card reflects the latest version. Any deviation creates a new gap item that is added to the work list.

Example: Data governance obligations (Article 10)

Article 10 requires providers to ensure the data used for training is appropriate and to document steps taken to address bias. We address these requirements through a combination of automated checks and manual documentation.

We run a data‑quality pipeline each night that scans the training set for missing values, duplicate entries, and label errors. The pipeline is built using open‑source components and is configured to fail the build if any critical threshold is exceeded. The pipeline logs are archived in our compliance bucket (path: gs://clauseguard-compliance/data-quality/).

For bias mitigation we maintain a living record that lists each intervention we have applied, such as re‑sampling under‑represented groups or adjusting loss weights. The record includes the date, the reason for the change, and the measured impact on fairness metrics. This record satisfies the documentation requirement and is reviewed by the ethics board before each model release.

We also refer to the NIST AI RMF NIST AI RMF when selecting controls for data governance. The framework’s categories (Govern, Map, Measure, Manage) help us verify that our pipeline covers all necessary aspects.

What this is NOT

This gap report is not a certificate of conformity. It does not guarantee that the AI system meets every requirement of the Act. It does not promise that regulators will accept the evidence we provide. It does not claim a 100 % success rate in finding every possible gap. It does not assert that the work list will never miss an issue. It is simply a transparent record of what we have done and what we still need to do, based on the sources we have consulted.

How we keep the report up to date

Maintaining an accurate gap report requires regular updates. We have instituted a lightweight rhythm that fits into our existing sprint cycle.

At the start of each sprint the compliance lead runs a script that compares the current version of the Act (downloaded from the EU’s official portal) with the master obligation list. Any new or changed sentences are added as fresh rows. The script also checks the evidence links to see if any have moved or become unavailable; broken links are flagged for the responsible owner.

During the sprint retrospective the team reviews the gap list and marks items as done when the associated evidence is uploaded and verified. Completed items are moved to an archive tab so they remain accessible for audits but do not clutter the active view.

Every quarter we hold a formal review meeting with legal, product, and engineering leads. In that meeting we walk through the high‑priority gaps, confirm effort estimates, and adjust timelines if needed. The outcome of the meeting is a short update note that is appended to the Confluence page.

Because the process is lightweight and tied to existing ceremonies we avoid creating extra overhead while still ensuring the report reflects the latest legal text and our current state of compliance.

References

- OWASP LLM Top 10 for Large Language Model Applications - NIST AI Risk Management Framework