The AI Readiness Assessment Framework: How to Score Your Company in 15 Minutes

AI readiness isn’t a feeling—it’s a measurable snapshot of how safely and effectively your organization can deploy, govern, and respond to AI systems in production. This 15-minute framework gives you a practical score across five dimensions that determine whether your AI efforts are scalable, defensible, and resilient.

What You’ll Need (Before You Start)

To complete the assessment quickly, gather:

• A list of AI systems or “agents” (chatbots, copilots, automated decision tools, RPA with LLMs, model APIs, internal ML services)
• Your security testing artifacts (threat models, test plans, scan results, red-team notes)
• Your governance approach (policies, monitoring dashboards, approval workflows)
• Your compliance materials (risk assessments, data maps, model cards, vendor documentation)
• Your incident response runbooks (on-call rota, escalation paths, past incident reports)

If you don’t have these handy, you can still score based on what you know exists today—just mark unknowns as gaps.

How Scoring Works (Simple and Consistent)

Score each dimension from 0 to 4, then add them for a total out of 20.

• 0 — Nonexistent: No formal process, ad hoc activity, or “we assume it’s fine.”
• 1 — Informal: Some effort exists, inconsistent, not documented, not repeatable.
• 2 — Basic: Documented and used sometimes; coverage is partial.
• 3 — Operational: Standardized, repeatable, used broadly; metrics exist.
• 4 — Mature: Continuously improved, audited, measurable; integrated into lifecycle and tooling.

Aim for speed: spend 3 minutes per dimension.

---

1) Agent Inventory Completeness

What it measures: Whether you can confidently answer: “Which AI systems are running, what do they do, who owns them, and what data do they touch?” If you can’t inventory it, you can’t govern it.

Score yourself (0–4)

• 0: No inventory; teams deploy AI tools independently.
• 1: A partial list exists (often in a spreadsheet) with missing owners or unclear scope.
• 2: Inventory includes most agents and key metadata, but isn’t updated reliably.
• 3: Central inventory is current and required for launches; ownership is clear.
• 4: Inventory is automated or enforced via workflows (e.g., intake forms, CI/CD gates); includes data classifications, vendors, and model versions.

Quick checklist (what “complete” looks like)

Your inventory should include, at minimum:

• Name and purpose (what decision or action the agent supports)
• Owner (business + technical)
• Users and access model (internal, customer-facing, privileged users)
• Data inputs/outputs (including sensitive categories)
• Model details (provider, version, fine-tuning, tools/plugins)
• Deployment context (where it runs, integrations, permissions)
• Risk tier (low/medium/high)

Actionable next step: Create a one-page intake template and require it before any AI system can access production data.

---

2) Security Testing Coverage

What it measures: How systematically you test AI systems for security risks such as prompt injection, data leakage, unsafe tool use, insecure integrations, and model supply chain issues.

Score yourself (0–4)

• 0: No AI-specific security testing; treated like a normal app with minimal review.
• 1: Occasional testing driven by individual engineers; no standard scenarios.
• 2: Standard test plan exists, but only applied to high-visibility projects.
• 3: Testing is required for production AI; coverage includes common AI abuse cases.
• 4: Continuous testing with regression suites; adversarial testing is routine; findings feed into engineering backlogs with SLAs.

What to test (fast but meaningful)

Prioritize coverage across:

• Prompt injection & data exfiltration (can it be tricked to reveal secrets?)
• Tool misuse (can it call internal systems in unsafe ways?)
• Authentication/authorization (least privilege for tools, scoped tokens)
• Data boundaries (PII handling, tenant isolation, logging exposure)
• Model/vendor risk (dependency review, version changes, permissions)

Actionable next step: Define a baseline “AI security test pack” (10–20 scenarios) and require it for every release.

---

3) Governance Monitoring Deployment

What it measures: Whether you monitor AI behavior in production—not just uptime, but quality, safety, and policy adherence.

Score yourself (0–4)

• 0: No monitoring beyond basic application logs.
• 1: Some dashboards exist, but not tied to governance policies or risk thresholds.
• 2: Monitoring is deployed for a subset of systems; alerts are inconsistent.
• 3: Standard monitoring exists for production agents; escalation paths are defined.
• 4: Monitoring is risk-tiered, includes automated policy checks, and is reviewed regularly with accountable owners.

What “governance monitoring” should include

• Input/output logging strategy (with privacy controls and retention rules)
• Policy enforcement signals (refusal rates, restricted-topic triggers, jailbreak attempts)
• Quality signals (user feedback, task success rates, hallucination reports)
• Drift and change management (model version changes, prompt changes, tool changes)
• Access and privilege monitoring (unusual tool calls, sensitive data access patterns)

Actionable next step: Pick 3–5 “must-alert” conditions for high-risk agents (e.g., suspected data leakage, abnormal tool calls) and wire them to on-call.

---

4) Compliance Documentation Quality

What it measures: Whether your AI systems have clear, consistent documentation that supports audits, internal approvals, and customer trust—and whether it reflects reality.

Score yourself (0–4)

• 0: No AI-specific documentation; approvals are informal.
• 1: Documentation exists for some systems but is incomplete, scattered, or outdated.
• 2: Standard templates exist (risk assessment, data mapping), but not consistently used.
• 3: Documentation is complete for production agents and updated with changes.
• 4: Documentation is lifecycle-managed, reviewed on a schedule, and tied to launch gates; evidence is easy to retrieve.

What “good documentation” includes

• System description (what it does and does not do)
• Data lineage (sources, destinations, retention, access controls)
• Risk assessment (intended use, misuse cases, impact analysis)
• Controls mapping (security tests, monitoring, human oversight)
• Third-party/vendor notes (roles, responsibilities, model updates)
• User disclosures and limitations (where applicable)

Actionable next step: Implement a “model/agent card” template and require it before procurement renewal or major releases.

---

5) Incident Response Capability

What it measures: Your ability to detect, contain, and recover from AI-related incidents—like sensitive data exposure, unsafe outputs, corrupted prompts, compromised tool access, or compliance violations.

Score yourself (0–4)

• 0: No AI incident plan; issues are handled ad hoc.
• 1: General incident response exists, but AI scenarios aren’t covered.
• 2: AI scenarios are documented, but drills are rare and responsibilities unclear.
• 3: Playbooks exist for key AI incident types; teams have practiced at least once.
• 4: Regular tabletop exercises; clear severity criteria; post-incident reviews drive hardening and policy updates.

Minimum viable AI incident playbooks

Create short playbooks for:

• Data leakage (logs, outputs, prompt history, connectors)
• Prompt injection leading to tool misuse
• Toxic/unsafe outputs in customer-facing channels
• Unauthorized model/provider changes
• Compliance-triggering events (e.g., processing prohibited data)

Actionable next step: Run a 30-minute tabletop exercise on “prompt injection causes unauthorized data access,” then update your playbook based on what breaks.

---

Calculate Your Score and Interpret It

Add your five dimension scores for a total out of 20.

• 0–7: High risk / low control

  • You’re likely to be surprised by AI behavior, data exposure, or audit requests.
  • Focus first on inventory + incident response to reduce existential risk quickly.

• 8–13: Emerging readiness

  • You have pockets of good practice but inconsistent coverage.
  • Standardize a baseline across teams: testing pack, templates, monitoring alerts.

• 14–17: Operational readiness

  • You can scale with confidence if you keep tightening lifecycle controls.
  • Invest in automation: gated releases, continuous testing, risk-tiered monitoring.

• 18–20: Mature readiness

  • Your controls are integrated and measurable.
  • Next step is optimization: shorten feedback loops, improve governance signal quality, and run regular drills.

Your 15-Minute Improvement Plan (Do This Next)

Pick the lowest-scoring dimension and commit to one concrete improvement within two weeks:

• If inventory is weak: launch an intake form + require owners and data classification.
• If security testing is weak: establish a baseline AI abuse-case test pack.
• If monitoring is weak: define must-alert signals and assign on-call ownership.
• If documentation is weak: standardize model/agent cards and store them centrally.
• If incident response is weak: write one playbook and run one tabletop drill.

AI readiness isn’t about perfection—it’s about repeatable control. Score yourself quarterly, track progress by dimension, and use the framework to turn AI governance from a vague initiative into a measurable capability.