{"author":{"@type":"Person","name":"Alexander Bock","url":"https://abxk.ai/en/autor/alexander-bock/"},"categories":["AI Security"],"content":{"markdown":"A threat model for one app, a red-team test for another and a policy that few people have read: that is not yet AI security. It becomes AI security when the pieces form a **program** — with an owner, a scope, a repeating cycle and evidence that the controls work. The gap is real. In IBM's 2025 breach study, 13% of organizations reported breaches of AI models or applications, and 97% of those had no AI access controls in place. Of all breached organizations in the study, 63% had no AI governance policy or were still developing one (Source: IBM and Ponemon Institute, Cost of a Data Breach Report 2025, ibm.com).\n\nThis is part 5, the last part of Level 4 and of our AI Security learning path. It brings the earlier parts together: the frameworks from part 1, the attack classes from part 2, the privacy techniques from part 3 and the evals from part 4. You will learn the eight components of a program, who owns what, how to judge your maturity, what to do in the first 90 days and which metrics show progress. The online shop of this level, with its four AI systems, serves as the example one last time.\n\n## A program, not a project\n\nA project ends. AI risks do not: providers release new models, teams adopt new tools, and attackers adapt. A program therefore works like a management system — the Plan-Do-Check-Act cycle that ISO/IEC 42001 uses, as described in [part 1 on governance frameworks](/en/ai-security/learn/ai-governance-frameworks/). You plan controls, run them, check whether they work and improve them.\n\nYou do not have to invent the content. Several well-known guides describe what belongs in it:\n\n- Google's **Secure AI Framework** (SAIF, 2023) names six core elements, from extending security foundations to AI to placing AI risks in the context of business processes.\n- The **Guidelines for secure AI system development**, published in 2023 by the UK's NCSC and the US agency CISA with partner agencies from other countries, cover four areas: secure design, development, deployment, and operation and maintenance.\n- The UK **Code of Practice for the Cyber Security of AI** (January 2025) sets 13 principles along the AI life cycle, from design to end of life.\n\nTogether with the NIST AI RMF from part 1 and with the OWASP Top 10 for LLM applications and MITRE ATLAS from earlier levels, they give you a shared vocabulary. The program's job is to turn that vocabulary into routines.\n\n## Eight components and their owners\n\nAn AI security program needs eight components. Each needs an owner and produces evidence:\n\n![Eight components of an AI security program in a grid: inventory, policy, risk assessment, secure development, monitoring, incident response, training and metrics, each with a short description; metrics is highlighted because it shows whether the others work](/images/learn/en/build-a-secure-ai-program-components.webp \"Eight components of an AI security program. Metrics show whether the other seven work.\")\n\n| Component | What it covers | Typical owner | Evidence |\n|---|---|---|---|\n| Inventory | Every AI system, model, tool and AI feature, including shadow AI | AI security lead | List with owner, data, role and risk class |\n| Policy | Approved tools, data rules, who decides | Management, with legal and data protection | Signed policy, list of approved tools |\n| Risk assessment | Threat model, impact assessment, AI Act classification | System owner, with security | Assessment per system, renewed after changes |\n| Secure development | Design reviews, secure RAG and agents, vetted components, evals as release gates | Engineering lead | Review records, eval results |\n| Monitoring | Logs of prompts, tool calls and decisions; alerts for abuse and drift | Security operations | Alert rules, dashboards, reviewed samples |\n| Incident response | AI-specific playbooks, roles, reporting duties | Security operations, with legal | Playbooks, exercise reports |\n| Training | Safe use for everyone, secure development for builders, risk decisions for leaders | HR, with the AI security lead | Training records by role |\n| Metrics | A few measures that show whether the program works | AI security lead | Quarterly report to management |\n\nFour components deserve a closer look. The **policy** should fit on one page: which tools are approved, which data may go into which tool, who approves a new AI use case and where people report problems. A short policy that people remember protects more than a long one that nobody reads. Review it at least twice a year, because the tools change quickly.\n\nThe **inventory** must include AI that nobody registered: tools that employees use on their own, AI features that vendors switch on in existing software, and models that teams download. Level 2 explains how to find this [shadow AI](/en/ai-security/learn/shadow-ai-in-companies/) without turning it into a blame game. **Incident response** needs its own playbooks, because AI incidents look different: a leaked system prompt, a poisoned knowledge base, an agent that acted on hidden instructions. Level 3 covers [AI incident response](/en/ai-security/learn/ai-incident-response/). Legal duties set deadlines here. Under the GDPR, a personal data breach must be reported to the authority within 72 hours where feasible. Under the AI Act, providers of high-risk systems must report serious incidents.\n\n**Monitoring** depends on the right logs. Level 3 listed what an agent should record and which properties logs need for an investigation. At program level, the questions are coverage and ownership. Does every system in the inventory send its logs to one central place, and can you at least export the logs of bought tools? Who reviews the alerts, and how fast? Then define alerts for the patterns from [part 2](/en/ai-security/learn/adversarial-machine-learning/): many near-identical queries from one account, sudden shifts in a model's decisions, or tool calls at unusual times.\n\nTools can support several components, but none of them is the program. According to the vendors, Airia governs AI agents, models, tools and data access with policies and audit evidence. Sentaro checks emails for phishing, which AI makes more convincing. Bitdefender protects business devices against ransomware, phishing and fileless attacks. Each one is a control inside the program, with an owner and a metric. More tools are in the [AI Security section](/en/ai-security/#tools).\n\n{{\u003c tools \"airia sentaro bitdefender\" \u003e}}\n\n## Where do you stand? A simple maturity model\n\nA maturity model helps you decide what to do next. The five stages below are a simplified model for orientation, not a standard:\n\n![A ladder with five maturity stages: ad hoc, aware, defined, measured and improving, each with what is typical at that stage; the online shop stands on stage two and aims for stage three](/images/learn/en/build-a-secure-ai-program-maturity.webp \"Five maturity stages, simplified. The online shop moves from stage 2 to stage 3 in the next quarter.\")\n\n1. **Ad hoc:** AI is in use, but nobody has an overview. Security depends on individuals.\n2. **Aware:** An inventory and a first policy exist, and owners are named.\n3. **Defined:** Every new AI system gets a risk assessment and follows a secure development standard. Incident playbooks exist.\n4. **Measured:** Evals, monitoring and red-team tests produce metrics, and management sees them regularly.\n5. **Improving:** Incidents, tests and metrics lead to changes in a steady cycle; an external audit, such as for ISO/IEC 42001, becomes realistic.\n\nTo find your stage, rate each of the eight components with four questions: Does it exist? Is it written down? Is it followed? Is it measured? Your overall stage is set by your weakest components, not your strongest.\n\nThe shop is at stage 2: it knows its four AI systems and has a policy, but only the support assistant went through a threat model. Its goal for the next quarter is stage 3. Do not skip stages. Metrics without defined processes measure chaos, and an audit before stage 3 mainly produces a list of gaps.\n\n## The first 90 days\n\nNinety days are a realistic period to move up one stage if you focus on a few systems. The plan below assumes a small team: a sponsor from management, one AI security lead and the owners of the main AI systems.\n\n![Timeline of the first 90 days in three phases: days 1 to 30 to get visibility, days 31 to 60 to assess and protect the most important systems, days 61 to 90 to test, measure and report, each with its main results](/images/learn/en/build-a-secure-ai-program-90-days.webp \"A 90-day plan in three phases. Each phase ends with a result that management can see.\")\n\n| Days | Focus | Main tasks | Result |\n|---|---|---|---|\n| 1–30 | Get visibility | Name sponsor and lead; build the inventory, including shadow AI; publish a one-page interim policy with approved tools; close obvious leaks, such as customer data in personal AI accounts | Inventory with owners and risk classes; interim policy |\n| 31–60 | Assess and protect | Threat models for the three most important systems; check vendor contracts for training on your data, retention and incident notice; add logging; start AI literacy training for all staff | Risk register; vendor review; logging in place |\n| 61–90 | Test, measure, report | First eval suite and red-team test for the riskiest system; incident playbook with a tabletop exercise; set baselines for the metrics; first report to management | Test results, playbook, metric baseline, plan for the next two quarters |\n\nWhich systems come first? Rank the inventory with four questions. Does the system make or prepare decisions about people? Does it touch personal or confidential data? Can it act without a human check, for example send emails or issue refunds? Can customers or the public reach it? Each yes raises the priority. To build the inventory itself, combine the discovery sources from the Level 2 part on shadow AI, and ask about use, not about rule-breaking.\n\nFor the shop, the riskiest system is the bought tool that pre-sorts job applications. It prepares decisions about people and is high-risk under the AI Act. So it gets the first full assessment, a review of the vendor's documentation and human oversight in the hiring team. The support assistant follows with the eval suite from [part 4](/en/ai-security/learn/evaluate-model-safety-and-reliability/).\n\n## Metrics that show whether the program works\n\nGood metrics are few, have an owner, show a trend and lead to a decision. Avoid counting activity, such as the number of policies written. Measure coverage and effect instead. Report to management every quarter in the same format: the inventory, the top risks with their owners, the metric trends, the incidents and the decisions you need. The same format makes trends visible and helps the sponsor act on them. These metrics are a good start:\n\n| Metric | What it shows | Shop example after 90 days |\n|---|---|---|\n| Inventory coverage | Share of known AI systems with owner and risk class | All four systems; two newly found tools added |\n| Assessment coverage | Share of high-risk and critical systems with a current assessment | Hiring tool and support assistant done |\n| Release gate | Share of AI releases that passed the eval suite before going live | Gate active for the support assistant |\n| Shadow AI | Unapproved tools found, and how many users moved to approved ones | Survey and proxy logs reviewed |\n| Training | Share of staff who finished role-based training | Course for all staff started |\n| Response | Time to detect and to contain AI incidents, including near misses | Measured in the tabletop exercise |\n\n{{\u003c callout type=\"try\" title=\"Write your one-page program charter\" \u003e}}\nOn one page, write down the sponsor and the lead, the AI systems in scope, and your current and target maturity stage. Add the three biggest risks from your inventory, the results you promise after 30, 60 and 90 days, and the four metrics you will report. Show the page to your sponsor this week. If they cannot say yes to it, the scope is too big.\n{{\u003c /callout \u003e}}","text":"A threat model for one app, a red-team test for another and a policy that few people have read: that is not yet AI security. It becomes AI security when the pieces form a program — with an owner, a scope, a repeating cycle and evidence that the controls work. The gap is real. In IBM\u0026rsquo;s 2025 breach study, 13% of organizations reported breaches of AI models or applications, and 97% of those had no AI access controls in place. Of all breached organizations in the study, 63% had no AI governance policy or were still developing one (Source: IBM and Ponemon Institute, Cost of a Data Breach Report 2025, ibm.com).\nThis is part 5, the last part of Level 4 and of our AI Security learning path. It brings the earlier parts together: the frameworks from part 1, the attack classes from part 2, the privacy techniques from part 3 and the evals from part 4. You will learn the eight components of a program, who owns what, how to judge your maturity, what to do in the first 90 days and which metrics show progress. The online shop of this level, with its four AI systems, serves as the example one last time.\nA program, not a project A project ends. AI risks do not: providers release new models, teams adopt new tools, and attackers adapt. A program therefore works like a management system — the Plan-Do-Check-Act cycle that ISO/IEC 42001 uses, as described in part 1 on governance frameworks. You plan controls, run them, check whether they work and improve them.\nYou do not have to invent the content. Several well-known guides describe what belongs in it:\nGoogle\u0026rsquo;s Secure AI Framework (SAIF, 2023) names six core elements, from extending security foundations to AI to placing AI risks in the context of business processes. The Guidelines for secure AI system development, published in 2023 by the UK\u0026rsquo;s NCSC and the US agency CISA with partner agencies from other countries, cover four areas: secure design, development, deployment, and operation and maintenance. The UK Code of Practice for the Cyber Security of AI (January 2025) sets 13 principles along the AI life cycle, from design to end of life. Together with the NIST AI RMF from part 1 and with the OWASP Top 10 for LLM applications and MITRE ATLAS from earlier levels, they give you a shared vocabulary. The program\u0026rsquo;s job is to turn that vocabulary into routines.\nEight components and their owners An AI security program needs eight components. Each needs an owner and produces evidence:\nEight components of an AI security program. Metrics show whether the other seven work. Build a Secure AI Program in Your Organization Component What it covers Typical owner Evidence Inventory Every AI system, model, tool and AI feature, including shadow AI AI security lead List with owner, data, role and risk class Policy Approved tools, data rules, who decides Management, with legal and data protection Signed policy, list of approved tools Risk assessment Threat model, impact assessment, AI Act classification System owner, with security Assessment per system, renewed after changes Secure development Design reviews, secure RAG and agents, vetted components, evals as release gates Engineering lead Review records, eval results Monitoring Logs of prompts, tool calls and decisions; alerts for abuse and drift Security operations Alert rules, dashboards, reviewed samples Incident response AI-specific playbooks, roles, reporting duties Security operations, with legal Playbooks, exercise reports Training Safe use for everyone, secure development for builders, risk decisions for leaders HR, with the AI security lead Training records by role Metrics A few measures that show whether the program works AI security lead Quarterly report to management Four components deserve a closer look. The policy should fit on one page: which tools are approved, which data may go into which tool, who approves a new AI use case and where people report problems. A short policy that people remember protects more than a long one that nobody reads. Review it at least twice a year, because the tools change quickly.\nThe inventory must include AI that nobody registered: tools that employees use on their own, AI features that vendors switch on in existing software, and models that teams download. Level 2 explains how to find this shadow AI without turning it into a blame game. Incident response needs its own playbooks, because AI incidents look different: a leaked system prompt, a poisoned knowledge base, an agent that acted on hidden instructions. Level 3 covers AI incident response. Legal duties set deadlines here. Under the GDPR, a personal data breach must be reported to the authority within 72 hours where feasible. Under the AI Act, providers of high-risk systems must report serious incidents.\nMonitoring depends on the right logs. Level 3 listed what an agent should record and which properties logs need for an investigation. At program level, the questions are coverage and ownership. Does every system in the inventory send its logs to one central place, and can you at least export the logs of bought tools? Who reviews the alerts, and how fast? Then define alerts for the patterns from part 2: many near-identical queries from one account, sudden shifts in a model\u0026rsquo;s decisions, or tool calls at unusual times.\nTools can support several components, but none of them is the program. According to the vendors, Airia governs AI agents, models, tools and data access with policies and audit evidence. Sentaro checks emails for phishing, which AI makes more convincing. Bitdefender protects business devices against ransomware, phishing and fileless attacks. Each one is a control inside the program, with an owner and a metric. More tools are in the AI Security section.\nAiriaSecurity \u0026amp; Governance · AI at the coreOrchestration platform for enterprises to build and manage AI solutions securely. SentaroSecurity \u0026amp; Governance · AI at the coreEmail security for small and mid-sized businesses that checks the context and intent of messages and warns of phishing. BitdefenderSecurity \u0026amp; Governance · Software with AI featuresCybersecurity for small and mid-sized businesses: protection against ransomware, phishing and fileless attacks. Short descriptions from our tool comparison, based on the vendors' information — not tested by ABXK.\nWhere do you stand? A simple maturity model A maturity model helps you decide what to do next. The five stages below are a simplified model for orientation, not a standard:\nFive maturity stages, simplified. The online shop moves from stage 2 to stage 3 in the next quarter. Ad hoc: AI is in use, but nobody has an overview. Security depends on individuals. Aware: An inventory and a first policy exist, and owners are named. Defined: Every new AI system gets a risk assessment and follows a secure development standard. Incident playbooks exist. Measured: Evals, monitoring and red-team tests produce metrics, and management sees them regularly. Improving: Incidents, tests and metrics lead to changes in a steady cycle; an external audit, such as for ISO/IEC 42001, becomes realistic. To find your stage, rate each of the eight components with four questions: Does it exist? Is it written down? Is it followed? Is it measured? Your overall stage is set by your weakest components, not your strongest.\nThe shop is at stage 2: it knows its four AI systems and has a policy, but only the support assistant went through a threat model. Its goal for the next quarter is stage 3. Do not skip stages. Metrics without defined processes measure chaos, and an audit before stage 3 mainly produces a list of gaps.\nThe first 90 days Ninety days are a realistic period to move up one stage if you focus on a few systems. The plan below assumes a small team: a sponsor from management, one AI security lead and the owners of the main AI systems.\nA 90-day plan in three phases. Each phase ends with a result that management can see. Build a Secure AI Program in Your Organization (2) Days Focus Main tasks Result 1–30 Get visibility Name sponsor and lead; build the inventory, including shadow AI; publish a one-page interim policy with approved tools; close obvious leaks, such as customer data in personal AI accounts Inventory with owners and risk classes; interim policy 31–60 Assess and protect Threat models for the three most important systems; check vendor contracts for training on your data, retention and incident notice; add logging; start AI literacy training for all staff Risk register; vendor review; logging in place 61–90 Test, measure, report First eval suite and red-team test for the riskiest system; incident playbook with a tabletop exercise; set baselines for the metrics; first report to management Test results, playbook, metric baseline, plan for the next two quarters Which systems come first? Rank the inventory with four questions. Does the system make or prepare decisions about people? Does it touch personal or confidential data? Can it act without a human check, for example send emails or issue refunds? Can customers or the public reach it? Each yes raises the priority. To build the inventory itself, combine the discovery sources from the Level 2 part on shadow AI, and ask about use, not about rule-breaking.\nFor the shop, the riskiest system is the bought tool that pre-sorts job applications. It prepares decisions about people and is high-risk under the AI Act. So it gets the first full assessment, a review of the vendor\u0026rsquo;s documentation and human oversight in the hiring team. The support assistant follows with the eval suite from part 4.\nMetrics that show whether the program works Good metrics are few, have an owner, show a trend and lead to a decision. Avoid counting activity, such as the number of policies written. Measure coverage and effect instead. Report to management every quarter in the same format: the inventory, the top risks with their owners, the metric trends, the incidents and the decisions you need. The same format makes trends visible and helps the sponsor act on them. These metrics are a good start:\nBuild a Secure AI Program in Your Organization (3) Metric What it shows Shop example after 90 days Inventory coverage Share of known AI systems with owner and risk class All four systems; two newly found tools added Assessment coverage Share of high-risk and critical systems with a current assessment Hiring tool and support assistant done Release gate Share of AI releases that passed the eval suite before going live Gate active for the support assistant Shadow AI Unapproved tools found, and how many users moved to approved ones Survey and proxy logs reviewed Training Share of staff who finished role-based training Course for all staff started Response Time to detect and to contain AI incidents, including near misses Measured in the tabletop exercise ▸Write your one-page program charter\nOn one page, write down the sponsor and the lead, the AI systems in scope, and your current and target maturity stage. Add the three biggest risks from your inventory, the results you promise after 30, 60 and 90 days, and the four metrics you will report. Show the page to your sponsor this week. If they cannot say yes to it, the scope is too big.\n","words":1859},"date":"2026-10-14","description":"The eight parts of an AI security program, from inventory and policy to monitoring and metrics, a simple maturity model and a plan for the first 90 days.","image":"https://abxk.ai/images/cover/build-a-secure-ai-program.webp","json":"https://abxk.ai/en/ai-security/learn/build-a-secure-ai-program/index.json","kind":"page","language":"en","learning_path":{"category":"ai-security","level":4,"level_name":"Expert"},"modified":"2026-10-14","related_tools":["Airia","Sentaro","Bitdefender"],"section":"ai-security","tags":["ai-security-program","ai-governance","security-metrics","maturity-model","expert"],"title":"Build a Secure AI Program in Your Organization","translations":[{"json":"https://abxk.ai/de/ai-security/learn/build-a-secure-ai-program/index.json","language":"de","url":"https://abxk.ai/de/ai-security/learn/build-a-secure-ai-program/"}],"url":"https://abxk.ai/en/ai-security/learn/build-a-secure-ai-program/"}