Two speech tools over MCP
Like the Language MCP server, the Azure Speech MCP server exposes Azure Speech through the Model Context Protocol — a host/client/server architecture whose advantage is dynamic tool discovery at runtime, so the agent picks tools from their descriptions with no routing logic.
Key concept · The two Speech tools
Speech-to-Text (Recognize) — convert an audio file to text; supports WAV, MP3, OGG, FLAC, MP4, M4A, AAC and more, with language selection, phrase hints, profanity filtering, and output-format options. Text-to-Speech (Synthesize) — convert text to natural neural-voice audio (e.g. en-US-JennyNeural, en-GB-SoniaNeural), output as WAV/MP3. The same six-step selection loop applies: user prompt → agent determines the task → matches a tool → calls it via the server → server runs Azure Speech → agent replies in natural language.
Blob storage and the SAS URL
Because the tools deal in audio files, an Azure Storage account is mandatory. For text-to-speech, the generated audio is written to Blob Storage and the agent's reply is a link to that file. For speech-to-text, the source can be a publicly accessible URL or a blob referenced by a SAS URL.
Key concept · Treat the SAS URL as a secret
You supply a SAS URL for a blob container granting the server read/write (plus add, create, list) permissions. Security discipline: use the shortest practical expiry, scope it to a single container, store it in a secret store with rotation, and never embed it in source code, prompts, or transcripts. Rotate immediately if exposure is suspected.
Portal wiring, then code
Set up a storage account and container, generate a SAS token (Read, Add, Create, Write, List), create a Foundry project and agent with a deployed model (e.g. gpt-4.1), then on the Tools page connect the Speech MCP server with the resource name, a bearer token (Ocp-Apim-Subscription-Key), and the X-Blob-Container-Url (the SAS URL). Test in the playground — "Generate '…' as speech" returns a file link; "Transcribe the file at …" accepts a public or SAS URL.
Key concept · Customization and code
The tools accept voice (e.g. en-GB-SoniaNeural), language (e.g. es-ES), phrase hints, and profanity filtering (masked, removed, or raw). In code you call the agent through the Responses API (azure-ai-projects + azure-identity), or wire the server yourself with an MCPTool (server_label, server_url ending in /speech/mcp, require_approval).
Chapter 20 quiz
Thirteen questions on the server, storage/SAS, and connecting it. Answer first, then expand Show answer.
Part A · Multiple choice
What two tools does the Azure Speech MCP server expose?
- A Recognize (STT) and Synthesize (TTS)
- B Detect and Redact
- C Translate and Summarize
- D Generate image and video
Show answer
A. Speech-to-text and text-to-speech.
Why is an Azure Storage account mandatory?
- A To store the model weights
- B Because the tools handle audio files (saved to / read from blobs)
- C To cache prompts
- D For billing only
Show answer
B. TTS writes audio to blob storage; STT reads from a URL/blob.
For text-to-speech, what does the agent's response contain?
- A The raw audio bytes inline
- B A link to the generated audio file in blob storage
- C A base64 image
- D Nothing
Show answer
B. A link to the file it saved.
Which is a recommended SAS-URL practice?
- A Shortest practical expiry, single-container scope
- B Never-expiring, account-wide
- C Embed it in the prompt
- D Commit it to source control
Show answer
A. Treat it as a secret; short expiry, minimal scope, rotation.
Which profanity-filtering options do the tools support?
- A masked, removed, or raw
- B on or off only
- C loud or quiet
- D none
Show answer
A — masked, removed, or raw.
Which header carries the SAS URL when connecting in the portal?
- A X-Blob-Container-Url
- B Authorization
- C Content-Type
- D User-Agent
Show answer
A — X-Blob-Container-Url.
Part B · True or false
Speech-to-text can transcribe from a public URL or a SAS-referenced blob.
Show answer
True. Both sources are supported.
The SAS token needs only Read permission.
Show answer
False. It needs Read, Add, Create, Write, and List.
The agent selects the speech tool autonomously from the tool descriptions.
Show answer
True. No manual routing logic is required.
You can only connect the Speech MCP server through the portal.
Show answer
False. You can define an MCPTool in code with the /speech/mcp server URL.
Part C · Short answer
Name three customization options the speech tools accept.
Show answer
Any three: voice, language, phrase hints, profanity filtering, output format.
Name three audio formats the recognizer supports.
Show answer
Any three: WAV, MP3, OGG, FLAC, MP4, M4A, AAC.
Which two packages build the client application?
Show answer
azure-ai-projects and azure-identity (Responses API).