Finding a model
The Foundry Models catalog (1,900+ models) has two commercial categories — sold directly by Azure (billed to your subscription) and from partners & community (own licensing). You filter by collection, capabilities (reasoning, tool calling, multimodal), source, inference task, fine-tuning method, and industry; each model's model card shows provider, capabilities, benchmarks, responsible-AI notes, and deployment options.
Key concept · Model types
- LLMs (GPT-5, Llama 3 70B) — deep reasoning at higher cost; SLMs (Phi-4, Llama 3 8B) — efficient, edge-friendly.
- Chat completion models (conversational) and reasoning models (math, coding, logic — show their working).
- Specialized: embedding (text→vectors for search/RAG), image / video generation, image analysis, text-to-speech / speech-to-text, and region/domain-specific models.
Selecting with objective data
Foundry organizes benchmarks into four dimensions — quality, safety, cost, and performance — surfaced through the model leaderboard and each model card's benchmarks tab.
| Dimension | Key metrics |
|---|---|
| Quality | Quality Index (0–1, averaged accuracy across datasets: reasoning, knowledge, QA, math, coding). |
| Safety | Harmful-behavior (HarmBench / Attack Success Rate — lower is safer), toxic content (ToxiGen F1), sensitive-domain knowledge (WMDP). |
| Cost | Cost per 1M input / output tokens; an estimated cost using a 3:1 input:output ratio. |
| Performance | Latency (mean, P50–P99), time to first token, throughput (generated / total tokens per second). |
Key concept · How to choose
Start with the quality index for a baseline, apply safety constraints, then optimize cost/speed. Use scenario leaderboards (reasoning, coding, math, QA, groundedness) if your app maps to one, read trade-off charts (quality vs cost/throughput — top-right is best on both), and finish with a side-by-side comparison of finalists across context window, features (function calling, vision), and supported endpoints. Interactive apps favor low latency; batch jobs favor cost.
From catalog to endpoint
Deployment makes a model callable through an endpoint. Foundry offers nine deployment types across three scopes plus a specialized one — the portal auto-selects the best fit.
| Scope | Types |
|---|---|
| Global (any region) | Global Standard (pay-per-token, highest quota — use when possible), Global Provisioned (reserved PTUs), Global Batch (50% discount, async). |
| Data Zone (EU/US) | Data Zone Standard / Provisioned / Batch — keeps data in a compliance zone. |
| Regional (single region) | Standard, Regional Provisioned — for regional data residency. |
| Specialized | Developer — fine-tuned model evaluation only. |
Key concept · Calling a deployment
Client apps need three things: the endpoint URL (a Foundry project endpoint or an OpenAI v1 endpoint), an authentication key (prefer Microsoft Entra ID in production over keys), and the deployment name (passed as the model parameter). Test code-free in the Playground — tuning temperature, max tokens, top-p, and system messages — before wiring it into an app.
Proving it performs
Evaluation guards quality, safety, and continuous improvement. It ranges from manual testing to automated metrics run over a dataset.
- Manual
- Interactive playground testing, structured human review (relevance, informativeness, accuracy, safety on a 1–5 scale), and user studies.
- Automated — generation quality
- Groundedness, relevance, coherence, fluency — often AI-assisted, with a GPT model as evaluator.
- Automated — safety
- Self-harm, hateful, violent, sexual, protected-material, jailbreak — reported as a defect rate.
- NLP metrics
- F1, BLEU, METEOR, ROUGE, GLEU — math-based, need ground-truth reference text.
Key concept · Iterate in order of cost
When scores fall short, improve in ascending cost: prompt engineering → a different model → RAG → fine-tuning. For safety concerns, add content filters (Azure AI Content Safety), prompt hardening, and output validation. Establish benchmarks early and re-run evaluations after every change so quality doesn't regress.
Chapter 2 quiz
Fifteen questions on the catalog, benchmarks, deployment, and evaluation. Answer first, then expand Show answer.
Part A · Multiple choice
What does the Quality Index measure?
- A Token cost
- B An averaged accuracy score (0–1) across benchmark datasets
- C Latency
- D Safety only
Show answer
B. It averages accuracy across reasoning, knowledge, QA, math, and coding — higher is better.
On the HarmBench safety benchmark, a lower Attack Success Rate means:
- A A less safe model
- B A safer, more robust model
- C Faster responses
- D Higher cost
Show answer
B. Lower ASR = the model resists harmful requests better.
Which deployment type should you use whenever possible for maximum capabilities?
- A Global Batch
- B Global Standard
- C Developer
- D Regional Provisioned
Show answer
B — Global Standard. Pay-per-token with the highest quota; the portal defaults to the best fit.
Which deployment type gives a 50% discount for large asynchronous jobs?
- A Provisioned
- B Batch
- C Standard
- D Developer
Show answer
B — Batch. Global/Data Zone Batch completes async jobs within 24 hours at half cost.
Which three things does a client app need to call a deployed model?
- A Endpoint URL, authentication key, and deployment name
- B GPU, RAM, and disk
- C A vector index and a prompt
- D Three models
Show answer
A. The deployment name is passed as the model parameter; prefer Entra ID over keys in production.
Which automated metric checks that responses are based on provided context rather than speculation?
- A Fluency
- B Groundedness
- C Coherence
- D BLEU
Show answer
B — Groundedness. Groundedness Pro gives a binary grounded/not-grounded assessment.
NLP metrics like BLEU and ROUGE require what to work?
- A A GPU cluster
- B Ground-truth reference text to compare against
- C A fine-tuned model
- D Nothing
Show answer
B. They're math-based comparisons, best where there's a definitive correct answer.
Part B · True or false
The estimated-cost benchmark combines input and output token costs using a typical 3:1 ratio.
Show answer
True. Three input tokens to one output token, as a single comparison number.
On a quality-vs-cost trade-off chart, models in the top-right corner excel at both metrics.
Show answer
True. Trade-off charts plot two metrics so you can find the best balance.
When evaluation scores fall short, fine-tuning is always the first thing to try.
Show answer
False. Improve in ascending cost: prompt engineering → different model → RAG → fine-tuning.
The Foundry Playground lets you test a deployment without writing code.
Show answer
True. It's an interactive chat interface with parameter tuning and a code tab.
Part C · Short answer
Name the four dimensions Foundry organizes benchmarks into.
Show answer
Quality, safety, cost, and performance.
Name the four generation-quality evaluation metrics.
Show answer
Groundedness, relevance, coherence, and fluency.
Distinguish an LLM from an SLM in one line each.
Show answer
LLM — deep reasoning and broad capability at higher cost; SLM — efficient, cost-effective, edge-friendly for common tasks.
Give three safety mitigations when safety metrics show concerns.
Show answer
Content filters (Azure AI Content Safety), prompt hardening (safety instructions in the system message), and output validation.