AI Platform Readiness: A Self-Assessment Across Nine Dimensions
Rate your AI platform across strategy, data, sovereignty, technology, economics, MLOps, identity, compliance, and skills. Four rungs per dimension, with the evidence that separates a claim from a fact.
This is the readiness model I use when assessing AI platforms. Nine dimensions, four rungs each. The rules of the exercise: rate what is true today, not the roadmap; every rating needs evidence you could show a skeptic; and a rating you cannot evidence drops a rung. That last rule is the whole game.
How to run it
Score each dimension 1 to 4. For each score, write the evidence in one line: a config, a dashboard, a log, a dated test. If the evidence is "we believe," the score is one lower. Regulated industries should weight sovereignty, identity, and compliance double; a platform is only as ready as its weakest dimension on the critical path.
1. Strategy
What it measures: whether AI work is pointed at named business outcomes.
| Rung | What is true |
|---|---|
| 1 | Experiments scattered across teams, no owner, success defined as "we tried it" |
| 2 | A use case list exists, prioritized by enthusiasm |
| 3 | Use cases tied to outcomes, with owners and kill criteria |
| 4 | Portfolio reviewed quarterly against measured outcomes; losers are actually killed |
Evidence to produce: the prioritized list, the owner names, the last kill decision with a date.
2. Data
| Rung | What is true |
|---|---|
| 1 | Data lives in wikis, inboxes, and someone's laptop |
| 2 | A central store exists; quality and lineage are folklore |
| 3 | A catalog with owners; quality checks on the datasets models touch |
| 4 | Lineage from source to model input; access is provisioned from the catalog, not by ticket |
Evidence: the catalog entry for the dataset behind your flagship model, with an owner who answers questions about it.
3. Sovereignty
| Rung | What is true |
|---|---|
| 1 | Data leaves for any provider with a signup page |
| 2 | A policy says where data may go; routing does not enforce it |
| 3 | Model traffic forced through a gateway; egress physically constrained; provider terms reviewed |
| 4 | Region, retention, and provider are properties of the network, tested quarterly |
Evidence: a failed curl from a workload subnet to a public endpoint, and the gateway log showing the same call succeeding through the chokepoint.
4. Technology
| Rung | What is true |
|---|---|
| 1 | Notebooks and scripts; production is a hope |
| 2 | Services in containers; deploys partly manual |
| 3 | Kubernetes with GitOps; environments reproducible from the repo |
| 4 | Progressive rollouts, tested rollback, infrastructure rebuilt from code in hours |
Evidence: the last rollback drill, dated, including the database-migration case.
5. Economics
| Rung | What is true |
|---|---|
| 1 | The AI line item is a monthly surprise |
| 2 | The bill decomposes by provider; no attribution to features |
| 3 | Cost per successful request by feature; budgets with alerts |
| 4 | Unit economics drive routing and model selection; budgets are code, reviewed with the infrastructure they cover |
Evidence: cost per request for your top three features, and the budget diff from the last PR that changed expected volume.
6. MLOps
| Rung | What is true |
|---|---|
| 1 | Prompts edited in production, models pinned by hope |
| 2 | Prompts and configs versioned; evals run before releases |
| 3 | Continuous evals in production; quality regressions alert |
| 4 | Eval gates block deploys and routing changes; regression history is queryable |
Evidence: the last time an eval gate stopped a change, with the diff it stopped.
7. Identity
| Rung | What is true |
|---|---|
| 1 | API keys in environment variables, shared across services |
| 2 | Keys in a secret store; still long-lived and shared |
| 3 | Workload identity (SPIFFE, IRSA, managed identity) per service; no shared credentials |
| 4 | Every model call attributable to a workload identity and an end-user context; unavoidable keys are short-lived and rotated |
Evidence: pick a model call from last Tuesday and show whose data it touched and which service made it.
8. Compliance
| Rung | What is true |
|---|---|
| 1 | "We will deal with it when legal asks" |
| 2 | Policies written; controls aspirational |
| 3 | Controls implemented as mechanisms (access, audit, transmission), mapped to the frameworks you answer to |
| 4 | Evidence generated continuously; an examiner's request is a query, not a project |
Evidence: the audit trail for one model decision, produced in under an hour.
9. Skills
| Rung | What is true |
|---|---|
| 1 | One person understands the AI systems; they are on vacation |
| 2 | A team exists; knowledge concentrated in its senior members |
| 3 | Runbooks current; on-call covers the AI platform; new engineers ship in weeks |
| 4 | The platform teaches itself: golden paths, templates, internal docs that match reality |
Evidence: the last incident handled by someone who did not build the system.
Reading your scores
- All 3s and 4s with evidence is rare. Treat it as suspicious and re-check the evidence.
- The gap between adjacent rungs is the work. Skip-a-rung plans fail: you do not get rung-4 economics without rung-3 attribution.
- Sequence by blast radius: sovereignty and identity gaps bound damage; economics and MLOps gaps compound cost; strategy gaps waste everything else.
- Re-run quarterly. Readiness decays as the platform, the team, and the regulatory landscape all move.
The honest output of this exercise is not a score. It is a ranked list of gaps between your rung and the next, each with evidence. That list is the roadmap.