← Architect AI Solutions for B… Latent Shelf · Ch.4

Module Study Guide

Evaluate Costs and Benefits of AI Solutions

TCO answers “what does this cost?” while ROI answers “what value does it deliver?” — both must be rigorous to justify an AI investment.

— the module's framing

The business-case module: define ROI criteria and total cost of ownership, build a defensible ROI analysis for a single process, decide build vs. buy vs. extend, and use a model router to trade cost against performance at runtime.

TCO vs ROI7-step analysisbuild/buy/extend13 quiz questions
01TCO & ROI criteria

Two complementary lenses

Total cost of ownership (TCO) sums every cost across the AI lifecycle; ROI quantifies the value delivered. Both are needed — organizations fail when they overestimate productivity gains, underestimate operating costs, or can't articulate value to leadership.

TCO components
Development, deployment, operational, change-management, and decommissioning costs — across the full lifecycle.
ROI benefit categories
Productivity gains, cost savings, revenue impact, risk reduction, and strategic/innovation value.

Key concept · Microsoft's ROI bar

AI ROI must be measurable, repeatable, aligned to business outcomes, and grounded in real usage analytics. Copilot Studio surfaces built-in metrics across four dimensions — usage (sessions, active users), automation (completion/abandonment/success rate), cost savings (time saved, savings per task), and quality (feedback scores, error & escalation rates) — so the business case rests on telemetry, not guesswork.

02Building an ROI analysis

Seven steps, grounded in one process

A defensible analysis scopes a single business process, establishes a “no-AI” baseline, and compares benefits against all TCO pillars:

1

Frame the scope

Name the process, set boundaries, capture baseline metrics (volume, average handle time, error rate, backlog, satisfaction).

2

Map value to levers

Time saved, quality lift, and capacity/throughput — each backed by a measurable observation.

3

Quantify savings

Use the savings calculator (per-run or per-tool): minutes saved × successful runs × fully-loaded labor rate.

4

Convert to financials

Annual benefit = (minutes/60 × runs/year × rate) + error cost avoided + backlog effects.

5

Model TCO

Five pillars: infrastructure, development & integration, data quality & prep, expertise & staffing, operations & licensing.

6

ROI, payback & sensitivity

ROI% = Net_Benefit ÷ Annual_TCO × 100; Payback = one-time cost ÷ net monthly benefit. Vary adoption, minutes saved, and rates across optimistic/expected/conservative cases.

7

Package the one-slide summary

Problem, intervention, measured impact, financials, confidence/risks, and a pilot-to-scale recommendation.

Worked example · Customer email triage

1.8 minutes saved per message × 50,000/month × 60% adoption at a $45/hr labor rate ≈ $486,000 annual savings against a Year-1 TCO of ~$300,000 → roughly 62% ROI and a ~7.4-month payback. Update the assumptions as analytics accrue to firm up the case.

03Build · Buy · Extend

Three pathways, one value-to-cost decision

Each AI capability can be built internally, bought as a prebuilt product, or extended from a mature platform (Copilot, Copilot Studio, Foundry). Score options on a weighted model across five cost domains and pick the best value-to-cost ratio:

Summary decision matrix.
FactorBuildBuyExtend
CustomizationMaximumMinimalHigh
Time-to-valueSlowestFastestMedium
CostHighestLowestMedium
Vendor dependencyNoneHighMedium
Maintenance burdenHighLowMedium

Quick reference: Build for competitive advantage + data sensitivity + capability + budget; Buy for speed + standard processes + limited internal capacity; Extend for a balance of speed and customization on a mature ecosystem.

04Model routing

Right model, right request

A model router (Azure AI Foundry) is a single unified endpoint that automatically directs each request to the most suitable model — LLM, small language model (SLM), or fine-tuned model — based on task type, cost, latency, and business rules, with centralized governance and monitoring.

Key concept · How routing optimizes cost & performance

Match complexity to model: simple → SLM, moderate → fine-tuned, complex → LLM — so simple tasks don't burn expensive LLM tokens. Rule types: static (if task = classification → SLM), weighted (proportional traffic for gradual rollout / A/B tests), fallback (auto-switch to a backup on failure), and version-based (route stable vs. experimental traffic). This makes “build, buy, or both” hybrid AI strategies practical with intelligent cost-performance trade-offs.

★Test yourself

Module 4 quiz

Thirteen questions on TCO/ROI, the analysis method, build-vs-buy-vs-extend, and model routing. Answer first, then expand Show answer.

Part A · Multiple choice

Q1

What does TCO capture that ROI does not?

  • A The value the solution delivers
  • B All costs across the AI lifecycle — “what does this cost?”
  • C The model's accuracy
  • D User satisfaction
Show answer

B. TCO sums lifecycle costs; ROI quantifies delivered value. Both are needed to justify the investment.

Q2

Which is NOT one of the five ROI benefit categories?

  • A Productivity gains
  • B Cost savings
  • C Token count
  • D Risk reduction
Show answer

C — Token count. The five are productivity gains, cost savings, revenue impact, risk reduction, and strategic/innovation value.

Q3

In the ROI analysis, what should be established first, before crediting any benefits?

  • A The final executive slide
  • B A “no-AI” baseline for the chosen process
  • C The model router config
  • D The vendor contract
Show answer

B. Scope a single process and set a steady-state baseline (volume, handle time, error rate) so all benefits are measured against it.

Q4

Which pathway generally offers the fastest time-to-value at the lowest cost but the highest vendor dependency?

  • A Build
  • B Buy
  • C Extend
  • D Retire
Show answer

B — Buy. Prebuilt products are fastest and cheapest to adopt but bring vendor lock-in and limited extensibility.

Q5

In a model router, a simple classification task should be routed to which model type?

  • A A large language model (LLM)
  • B A small language model (SLM)
  • C A multi-modal model
  • D Whichever is most expensive
Show answer

B — SLM. Match complexity to model (simple → SLM, moderate → fine-tuned, complex → LLM) so simple tasks don't burn expensive LLM tokens.

Q6

Which routing rule type distributes traffic proportionally to enable gradual rollouts and A/B testing?

  • A Static routing
  • B Weighted routing
  • C Fallback routing
  • D Version-based routing
Show answer

B — Weighted routing. Fallback switches to a backup on failure; version-based separates stable from experimental traffic.

Part B · True or false

Q7

Microsoft's guidance says AI ROI should be measurable, repeatable, aligned to business outcomes, and grounded in real usage analytics.

Show answer

True. The business case should rest on telemetry (e.g. Copilot Studio analytics), not estimates alone.

Q8

Sensitivity analysis (optimistic/expected/conservative cases) is unnecessary once you have a single point estimate.

Show answer

False. Ranges show upside and risk tolerance, grounded in your own telemetry — decision-makers need the bands, not just one number.

Q9

A model router requires a separate integration for each underlying model.

Show answer

False. It's a single unified endpoint fronting multiple models, which reduces code complexity and centralizes governance.

Q10

“Extend” sits between Build and Buy, balancing customization with speed on a mature ecosystem.

Show answer

True. Extending Copilot/Copilot Studio/Foundry offers more customization than Buy and faster deployment than Build.

Part C · Short answer

Q11

Name the five TCO components across the AI lifecycle.

Show answer

Development, deployment, operational, change-management, and decommissioning costs. (In the analysis method, TCO is modeled as five pillars: infrastructure, dev & integration, data quality & prep, expertise & staffing, operations & licensing.)

Q12

Give the simple ROI and payback formulas.

Show answer

ROI% = (Net Benefit ÷ Annual TCO) × 100, where Net Benefit = Annual Benefit − Annual TCO; Payback (months) = one-time cost ÷ net monthly benefit.

Q13

Name three benefits a model router provides beyond cost savings.

Show answer

Any three of: performance (route latency-sensitive or domain-specific tasks to the right model), reliability (automatic failover), centralized governance/monitoring, gradual rollouts & A/B testing (weighted routing), and version management.