> For the complete documentation index, see [llms.txt](https://www.aicontrolarchitecture.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.aicontrolarchitecture.org/the-ten-pillars/14-pillar-assurance-and-testing.md). # 14 · Assurance & Testing **Control question:** *Do the controls actually hold?* **Surface:** proves the boundedness of See, Decide, and Do. *** ## Purpose A control that has never been tested is a *claim*, not a control. This pillar makes control strength **provable**: it requires that a use case's controls be tested proportionate to its tier, that high-tier controls face adversarial testing, and that results are retained as evidence and re-run when the use case changes. It is how a boundary moves from *declared* or *evidenced* to *verified*. *** ## Why it matters AI controls fail quietly. A data boundary looks configured but leaks under a certain query; an injection filter blocks the obvious cases and misses the crafted one; an action allow-list is bypassed through an unexpected tool path. None of this shows up until someone looks, deliberately, adversarially, before an incident does the looking for you. Assurance is the pillar that turns "we have controls" into "we have *tested* controls," and it is the difference between a architecture that documents intentions and one that can survive its own show-me. *** ## Control objectives * **Test controls before go-live**: proportionate to tier. * Subject high-tier use cases to **adversarial testing** of input, output, and action controls. * **Retain results** as evidence and **re-run** on material change. *** ## Requirements From the [Requirements Catalogue](/foundation/06-requirements-catalogue.md): | ID | Requirement | From | Boundary | | --------- | -------------------------------------------------------------------------------------- | ---- | -------- | | ACA-14-01 | Controls are tested before go-live, proportionate to tier. | T2 | Verified | | ACA-14-02 | High-tier use cases undergo adversarial testing of input, output, and action controls. | T3 | Verified | | ACA-14-03 | Test results are retained as evidence and re-run on material change. | T2 | Verified | *** ## Key controls * **Control test plan**: for each in-scope control, a defined way to test that it holds, and the expected result (see the [AI Assurance Test Plan template](https://github.com/ai-ankqush/ai-control-architecture/tree/main/templates/ai-assurance-test-plan-template.md)). * **Pre-go-live gate**: controls are tested before an elevated-tier use case is allowed live; failures block or tier-down. * **Adversarial / red-team testing**: for high-tier use cases, actively try to break input, output, and action controls (prompt injection, boundary escape, unauthorized action) rather than only confirming happy paths. * **Regression on change**: model updates, new tools, new data sources, or scope changes re-trigger the relevant tests; a control verified last quarter is not assumed to hold today. * **Evidence capture**: results are recorded as the *verified* boundary evidence the other pillars point to. The how-to for running assurance and audit lives in the operating guide ([Assurance & Audit Guide](/docs-readme.md)); this pillar defines what assurance is *required*. *** ## Tier guidance | | T1 | T2 | T3 | | -------------------------- | ----------- | ----------- | -------- | | Pre-go-live control test | Recommended | Required | Required | | Adversarial testing | - | Recommended | Required | | Retain results as evidence | - | Required | Required | | Re-test on material change | - | Required | Required | *** ## Evidence The [AI Assurance & Testing template](https://github.com/ai-ankqush/ai-control-architecture/tree/main/templates/ai-assurance-and-testing-template.md) and [AI Assurance Test Plan template](https://github.com/ai-ankqush/ai-control-architecture/tree/main/templates/ai-assurance-test-plan-template.md) capture the test plan, execution, and results. This pillar is what *produces* the *Verified* boundary source that pillars 08–12 rely on, assurance is how their controls earn that grade. *** ## Standards crosswalk Maps to NIST AI RMF **Measure**, ISO/IEC 42001 (verification, validation, performance evaluation), EU AI Act (testing, accuracy, robustness, Arts. 9, 15), and SR 11-7 (model validation and effective challenge). See [`mappings/`](/standards-crosswalks/mappings-readme.md). *** ## Failure modes * Controls documented but never tested, claims dressed as controls. * Happy-path testing only; no adversarial attempt to break them. * A model or tool change that silently invalidates a previously verified control. * Test results that exist but were never captured as evidence. *** **Next:** [15 · Monitoring & Evidence](/the-ten-pillars/15-pillar-monitoring-and-evidence.md) *Version 0.1.0 · Licensed under the terms in* [*LICENSE.md*](/project/license.md)*.* --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://www.aicontrolarchitecture.org/the-ten-pillars/14-pillar-assurance-and-testing.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.