Five things AI apps do
Modern AI solutions are built on machine learning models that let apps interpret input, reason over data, and generate responses. As a developer you draw on five capability areas — identifying which you need tells you which services to provision.
| Capability | What it does |
|---|---|
| Generative AI & agents | LLMs that generate original responses to prompts; agents add instructions + tools to automate tasks. |
| Natural language processing | Statistical/semantic understanding of text — analysis, classification, sentiment, summarization. |
| Computer speech | Recognizes and synthesizes speech — conversation, transcription, translation, read-aloud. |
| Computer vision | Interprets images, video, and camera streams; multimodal models also generate visuals. |
| Information extraction | Combines generative AI, NLP, vision, and speech to pull key data from documents, images, and recordings. |
Key concept · Capability drives service choice
The point of naming capabilities is practical: an expense-claim app needs information extraction (dates, line items, costs from receipts); a checkout app needs computer vision; a support bot needs generative AI plus speech. Match capability → Azure service before you build.
The platform and its projects
Microsoft Foundry is the recommended platform for AI development on Azure — you work through the Foundry portal (web UI) or the Foundry SDK (programmatic). Work is organized into projects: each project belongs to a single Foundry resource that supplies compute, storage, and AI tools, and one project is the default.
Key concept · A project's four asset types
- Models — LLM deployments from the Foundry Models catalog (Microsoft, OpenAI, others), reached via Foundry APIs/SDKs or the Azure OpenAI endpoint.
- Agents — named configs bundling an LLM + instructions + tools, run through the Foundry Agent service.
- Tools — built-in (web search, code interpreter), custom/third-party via MCP connections, and the Foundry Tools suite.
- Knowledge — Foundry IQ creates one central MCP-based connection to multiple knowledge sources for grounding.
The portal lets developers find, compare, deploy, and test models; create and test agents; wire up MCP tools and Foundry IQ knowledge; manage access; and obtain the endpoints and API keys client apps need. The Foundry SDK does the same programmatically for CI/CD. (An older hub-based "classic" architecture still exists; this course uses the current project architecture.)
Prebuilt services and the developer toolchain
Foundry Tools (formerly Azure AI Services / Cognitive Services) are prebuilt APIs offering common AI functions — often more cost-effective and predictable than building on generative agents alone.
| Tool | Purpose |
|---|---|
| Azure Language | Entity extraction, sentiment, summarization, conversational language, question answering. |
| Azure Speech | Text-to-speech, speech-to-text, real-time live speech. |
| Azure Translator | Translate text across many languages. |
| Azure Document Intelligence | Extract fields from invoices, receipts, and forms (prebuilt + custom models). |
| Azure Content Understanding | Multimodal analysis of documents, images, video, and audio. |
Key concept · The toolchain
Client apps connect to tool endpoints with a project key or token-based auth. Build in VS Code (with the Foundry Toolkit extension — browse resources, deploy models, test in playgrounds, author agents visually or in YAML, generate integration code) or Visual Studio, using the Foundry SDK (Foundry assets), the OpenAI API (chat on Foundry models with OpenAI syntax), or Foundry Tools SDKs / REST APIs. Azure AI development is language-agnostic — C#, Python, Node.js, TypeScript, Java — with GitHub and Copilot for source control and productivity.
Six principles, across the lifecycle
Because AI's human-like output invites implicit trust, Microsoft applies six responsible-AI principles throughout development — tooling alone is not enough; each must be considered from the start.
| Principle | What it requires |
|---|---|
| Fairness | Treat all people fairly; no bias by gender, ethnicity, or other factors. Review training data for representativeness. |
| Reliability & safety | Perform reliably with minimal risk; rigorous testing and confidence thresholds for high-stakes uses. |
| Privacy & security | Protect personal data in training and inference; safeguard customer content. |
| Inclusiveness | Empower everyone; involve diverse perspectives in design, development, and testing. |
| Transparency | Make purpose, workings, and limitations understandable; disclose data use and confidence. |
| Accountability | People — not machines — are accountable; work within governance and legal frameworks. |
Chapter 1 quiz
Fourteen questions on the AI capabilities, Microsoft Foundry, Foundry Tools and SDKs, and responsible AI. Answer first, then expand Show answer.
Part A · Multiple choice
Which capability combines generative AI, NLP, vision, and speech to pull key data from receipts and forms?
- A Computer vision
- B Information extraction
- C Computer speech
- D Natural language processing
Show answer
B — Information extraction. It layers multiple AI technologies to extract fields from documents, images, and recordings.
What are the three things a Foundry agent bundles together?
- A A model, a database, and a key
- B An LLM, instructions, and tools
- C A prompt, a GPU, and a REST API
- D Vision, speech, and translation
Show answer
B. An agent is a named configuration of an LLM + instructions + tools, run via the Foundry Agent service.
What does Foundry IQ provide?
- A A model catalog
- B A single central MCP-based knowledge connection to multiple sources
- C A billing dashboard
- D A code interpreter
Show answer
B. Foundry IQ simplifies grounding by unifying knowledge sources through one MCP connection.
Which Foundry Tool extracts fields from invoices and receipts?
- A Azure Language
- B Azure Translator
- C Azure Document Intelligence
- D Azure Speech
Show answer
C — Azure Document Intelligence. Prebuilt and custom models extract fields from complex documents.
What was "Foundry Tools" previously called?
- A Azure AI Services / Azure Cognitive Services
- B Azure Machine Learning
- C Power Platform
- D Azure DevOps
Show answer
A. APIs and SDKs may still reference the older names.
Which VS Code extension streamlines Foundry development (browse resources, deploy models, author agents)?
- A The Foundry Toolkit extension
- B GitHub Copilot
- C Azure Functions Core Tools
- D The OpenAI plugin
Show answer
A — Foundry Toolkit extension. It offers playgrounds, a visual agent designer, YAML config, and code generation.
Which responsible-AI principle says people, not machines, bear responsibility for AI systems?
- A Transparency
- B Accountability
- C Fairness
- D Inclusiveness
Show answer
B — Accountability. Responsibility rests with those who build and deploy the system, within a governance framework.
Part B · True or false
Each Microsoft Foundry project belongs to a single Foundry resource.
Show answer
True. A Foundry resource can host one or more child projects, one of which is the default.
Custom and third-party tools are connected to agents through Model Context Protocol (MCP) connections.
Show answer
True. Alongside built-in tools like web search and code interpreter.
Azure AI development on Foundry is limited to a single programming language.
Show answer
False. It's language-agnostic — C#, Python, Node.js, TypeScript, Java, and more, via SDKs or REST APIs.
Prebuilt Foundry Tools can be more cost-effective and predictable than building everything on generative agents.
Show answer
True. They offer common AI functions out of the box as an alternative to generative-only solutions.
Responsible-AI tooling alone is enough; fairness need not be considered until after deployment.
Show answer
False. Fairness (and every principle) must be considered from the start, across the whole lifecycle.
Part C · Short answer
Name the four asset types a Microsoft Foundry project organizes.
Show answer
Models, agents, tools, and knowledge.
Name the six responsible-AI principles.
Show answer
Fairness, reliability & safety, privacy & security, inclusiveness, transparency, and accountability.