Ch 1 — AI Strategy¶

Learning Objectives¶

By the end of this chapter you will be able to:

1. Place an organisation on a five-level AI maturity model and recommend the highest-leverage investments for each transition.
2. Apply a business impact vs technical feasibility matrix to prioritise an AI use-case portfolio with concrete examples.
3. Calculate AI project ROI — including cost savings, revenue uplift, and total cost of ownership — and derive a payback period.
4. Use a structured build vs buy vs partner decision tree to make a defensible technology selection.
5. Define phase-gate success criteria for each stage of the AI project lifecycle from Discovery through Scaling.

AI Maturity Model¶

Most organisations sit between the extremes of "no intentional AI use" and "AI embedded in every decision at scale." Understanding where you are determines what investments are highest leverage — and prevents cargo-culting practices from organisations at a different maturity level.

Progression levers:

• Level 1 → 2: Identify one high-value use case, assign a dedicated ML engineer, establish Git + experiment tracking (MLflow), and run a time-boxed PoC.
• Level 2 → 3: Build shared infrastructure (feature store, model registry, CI/CD for models); establish coding standards; define monitoring SLOs.
• Level 3 → 4: Align AI roadmap to business OKRs; instrument every model with business outcome metrics; automate the model refresh pipeline.
• Level 4 → 5: Automate evaluation gates in CI/CD; build an AI governance function; publish model cards for all production models; establish a red team.

Use Case Selection Framework¶

Not all AI use cases are equal. A 2×2 matrix of Business Impact (Y-axis) vs Technical Feasibility (X-axis) provides a first-pass prioritisation that prevents two common failure modes: pursuing technically impressive but low-value projects, and underestimating feasibility barriers.

                     HIGH BUSINESS IMPACT
                              │
    Funded Research           │    Prioritise Now
    ─────────────────         │    ───────────────
    High value,               │    High value,
    hard to build             │    achievable today
    → invest in prerequisites │    → execute first
                              │
  LOW ───────────────────────-┼──────────────────── HIGH
  FEASIBILITY                 │                  FEASIBILITY
                              │
    Deprioritise              │    Quick Wins
    ────────────              │    ──────────
    Low value,                │    Low value,
    hard to build             │    easy to build
    → avoid                   │    → nice-to-have
                              │
                      LOW BUSINESS IMPACT

Concrete examples by quadrant:

Scoring feasibility (score each 1–5, sum to rank):

ROI Calculation¶

Single-number ROI calculations that ignore ongoing costs systematically overstate returns. The following formula covers all material value and cost components:

Worked example — AI-powered invoice processing:

Build vs Buy vs Partner Decision Framework¶

This is one of the highest-stakes strategic decisions in enterprise AI. The default has shifted as foundation model APIs have made "buy (API + fine-tuning)" viable for capabilities that previously required full custom builds.

flowchart TD
    A["New AI capability needed"] --> B{"Core competitive\ndifferentiator?"}
    B -->|"Yes"| C{"Sufficient data\nand ML talent?"}
    B -->|"No"| D{"Vendor covers\n≥ 80% of needs?"}
    C -->|"Yes"| BUILD["BUILD\nPropriety model on\nown data and infrastructure"]
    C -->|"No"| PARTNER["PARTNER\nJoint build or acqui-hire\nto close talent gap"]
    D -->|"Yes"| BUY["BUY (SaaS/API)\nVendor product;\nminimise build effort"]
    D -->|"No"| E{"Foundation model +\nfine-tuning adequate?"}
    E -->|"Yes"| BUY_FM["BUY (Foundation model API)\nFine-tune on own data;\nprompt engineering"]
    E -->|"No"| PARTNER

Total Cost of Ownership¶

TCO analysis over a 3-year horizon consistently reveals that talent and maintenance dominate compute costs:

LLM API cost modelling:

"""
tco.py — LLM API cost modelling and 3-year TCO projection.

Requirements: pip install anthropic
"""
from __future__ import annotations


def monthly_llm_cost(
    daily_requests: int,
    mean_input_tokens: int,
    mean_output_tokens: int,
    input_price_per_million: float,
    output_price_per_million: float,
) -> dict[str, float]:
    """
    Estimate monthly LLM API cost.

    Args:
        daily_requests:             Number of API calls per day.
        mean_input_tokens:          Average input tokens per call.
        mean_output_tokens:         Average output tokens per call.
        input_price_per_million:    Vendor price per million input tokens (USD).
        output_price_per_million:   Vendor price per million output tokens (USD).

    Returns:
        Dict with token volumes and cost breakdown.
    """
    monthly = daily_requests * 30
    in_tokens = monthly * mean_input_tokens
    out_tokens = monthly * mean_output_tokens
    input_cost = in_tokens * input_price_per_million / 1_000_000
    output_cost = out_tokens * output_price_per_million / 1_000_000
    return {
        "monthly_requests": monthly,
        "monthly_input_tokens": in_tokens,
        "monthly_output_tokens": out_tokens,
        "input_cost_usd": round(input_cost, 2),
        "output_cost_usd": round(output_cost, 2),
        "total_cost_usd": round(input_cost + output_cost, 2),
    }


# Example: 5,000 requests/day, 1,200 input tokens, 600 output tokens (Haiku pricing)
cost = monthly_llm_cost(
    daily_requests=5_000,
    mean_input_tokens=1_200,
    mean_output_tokens=600,
    input_price_per_million=0.80,
    output_price_per_million=4.00,
)
print(f"Monthly LLM API cost: ${cost['total_cost_usd']:,.2f}")
# Monthly LLM API cost: $1,584.00

AI Project Lifecycle Phases¶

Phase gates must have real veto power. Pressure to skip gates — especially PoC → Pilot — is the single most reliable predictor of expensive production incidents.

Common Failure Modes¶

Measuring Business Impact¶

Leading vs Lagging Indicators¶

Monitor leading indicators weekly; lagging indicators per experiment cycle. Neither alone is sufficient: leading metrics can improve while business outcomes stagnate (goodhart's law), and lagging metrics appear too late to catch fast-moving regressions.

Attribution Challenges¶

• Confounding factors: seasonal effects, marketing campaigns, and product changes co-occur with model deployments.
• Selection bias: users who interact with the AI feature may already be higher-intent.
• Long feedback loops: churn reduction may not be visible for 90 days post-deployment.

Best practice: Randomised A/B tests with pre-registered primary metrics and minimum four-week baseline measurement. Observational correlation does not establish causation.

Exercises¶

1. Maturity assessment: Apply the five-level maturity model to an organisation you know (or a publicly described case study). Write a two-page diagnosis: (a) current level with three pieces of evidence, (b) the top two investments to reach the next level, © the single biggest blocker and how to remove it.

2. Use case portfolio: Generate six AI use cases for a mid-sized retail bank. Place each on the 2×2 matrix with a one-paragraph feasibility justification. Recommend which two to pursue first and explain the sequencing rationale.

3. ROI model: Build a Python ROICalculator dataclass that accepts monthly value components (cost savings, revenue uplift), monthly run costs (infra, talent, tooling, compliance), and upfront build cost. Implement net_annual_value(), roi_pct(), and payback_months() methods. Run it for the invoice processing example and for two additional hypothetical use cases.

4. Build vs buy analysis: A healthcare company wants AI to extract structured data from clinical notes. Evaluate three options: (a) fine-tune a medical LLM on proprietary EHR data, (b) use a general-purpose foundation model API with prompt engineering, © purchase a specialised medical NLP SaaS. Compare on: data privacy, customisation depth, time to market, regulatory compliance burden, and 3-year TCO. Make a recommendation using the decision tree.

5. Phase gate checklist: Design a one-page gate review checklist for the Pilot → Production transition for an LLM-based credit decisioning system (EU AI Act high-risk). Include: minimum performance thresholds (with numbers), required documentation, required stakeholder approvals, monitoring requirements that must be active before go-live, and rollback criteria.

Summary¶

• AI maturity (Level 1 Ad Hoc → Level 5 AI-Native) provides a diagnostic lens for where to invest; most organisations at Level 2–3 benefit most from shared MLOps infrastructure and cross-functional process improvements before adding more models.
• Use case selection via Business Impact × Technical Feasibility 2×2 prevents the twin failure modes of technically impressive but low-value projects and chronically underestimated feasibility barriers.
• ROI calculation must include all TCO components over a multi-year horizon; the 20% of build cost annual maintenance cost is the most consistently omitted line item.
• Build vs buy vs partner is a structured decision driven by competitive differentiation, talent availability, data privacy, and time to value — the default has shifted toward "buy (foundation model API)" for non-differentiating capabilities.
• Phase gates (Discovery → PoC → Pilot → Production → Scaling) must have real veto power; the PoC → Pilot gate is where most enterprise AI projects fail when data quality assumptions prove false.
• Business impact measurement requires randomised A/B tests with holdout groups for causal attribution; both leading (model quality) and lagging (business outcome) indicators are necessary for comprehensive signal coverage.

Next: Ch 2 — Enterprise AI Architecture