Agentic voice best practices

Amazon Connect agentic voice combines advanced speech recognition (ASR) with expressive, natural-sounding voices to deliver a complete voice AI experience. Advanced ASR provides natural, low-latency turn taking in real-time conversations, while the voice engine delivers lifelike speech across 50+ locales. Together, these components power seamless, human-like interactions between callers and Amazon Connect AI agents. This guide covers configuration and best practices for both components, so you can get the most out of the full agentic voice experience.

This guide covers:

Advanced speech recognition (ASR) best practices

The Advanced ASR speech model preference is a streaming recognizer designed for natural, low-latency turn taking in real-time conversations. Instead of relying only on a fixed window of silence, it evaluates the caller's speech as they talk and predicts when the caller has finished — the end of turn (EOT).

Session attribute syntax

The controls in this section are set with standard Lex V2 session attributes in the x-amz-lex namespace. You set them when you start a conversation and can override them in a Lambda function (for example, to relax detection only while collecting a sensitive value).

Scope every attribute to an intent and slot:

x-amz-lex:audio:<setting>:<intentName>:<slotToElicit>

Do:

Don't:

Barge-in (interrupt behavior)

Barge-in lets the caller interrupt a prompt the bot is playing. It is enabled by default, which is usually what you want for natural conversation. You control it with:

x-amz-lex:allow-interrupt:<intentName>:<slotToElicit>

Default: true

Disable barge-in (false) when:

Keep barge-in enabled when:

End-of-turn tuning

Advanced ASR ends a turn when either condition is met first:

Higher values make the bot wait longer and end turns more conservatively (fewer premature cutoffs, slightly more latency). Lower values end turns sooner (lower latency, higher chance of cutting off a caller who pauses). The confidence threshold is the primary lever; the silence timeout only matters when confidence has not already ended the turn.

Out-of-range behavior:

Choosing values

Example

Conservative end-of-turn detection for the AccountNumber slot of the VerifyIdentity intent:

{
  "sessionState": {
    "sessionAttributes": {
      "x-amz-lex:audio:end-confidence-threshold:VerifyIdentity:AccountNumber": "0.9",
      "x-amz-lex:audio:end-timeout-ms:VerifyIdentity:AccountNumber": "3000"
    }
  }
}

A single default for every intent and slot:

{
  "sessionState": {
    "sessionAttributes": {
      "x-amz-lex:audio:end-confidence-threshold:*:*": "0.8",
      "x-amz-lex:audio:end-timeout-ms:*:*": "800"
    }
  }
}

Handling long-running tool calls

When an AI agent or Lambda makes a long-running tool call (a backend lookup, an external API, a model invocation) before the bot is ready to listen, the caller sits in silence. To the caller this can feel like the bot stalled or talked over them when it finally responds.

Mitigations:

ASR session attribute quick reference

Common ASR mistakes

Voice best practices

Amazon Connect agentic voice delivers expressive, natural-sounding text-to-speech across 50+ locales. The engine automatically normalizes written text into natural speech. In most cases, you can pass text as-is and get great results without any special formatting.

Text formatting

General rules

Pass natural, well-punctuated text. Full sentences with normal capitalization and punctuation produce the best pacing and intonation.

Do:

Don't:

Automatic normalization

The engine reads common written formats as natural speech. Most times, you don't need to reformat them:

Speech control tags

The voice engine automatically infers appropriate pacing and emotion from transcript content. Use control tags only for specific use cases where the default does not satisfy. All tags are placed directly in the transcript.

Speed

Control the speaking rate. Range: 0.6 (slow) to 1.5 (fast). Default: 1.0.

<speed ratio="0.85"/>This call may be recorded for quality purposes.

To reset speed after slowing down:

<speed ratio="0.85"/>Important notice.<speed ratio="1.0"/>Now, how can I help?

Volume

Control loudness. Range: 0.5 (quiet) to 2.0 (loud). Default: 1.0.

<volume ratio="0.5"/>I'll speak softly for this part.

Breaks

Insert pauses. Specify duration in seconds (s) or milliseconds (ms).

Your balance is $1,234.<break time="500ms"/>Your next payment is due June 15th.

Use breaks for:

Spell

Force character-by-character readout for codes, IDs, phone numbers, or any string that should be spoken letter-by-letter.

Your confirmation code is <spell>TKT4829XB</spell>.

For long sequences, add a space inside the spell tag wrapper to insert pauses:

Your phone number is <spell>(415)</spell><break time="200ms"/><spell>5551212</spell>.

Alternative formats (without tags):

Emotion

The model naturally infers emotional tone from content — no tags needed in most cases. Emotion tags are a beta capability and work most reliably when the emotion matches the transcript.

<emotion value="sympathetic"/>I understand this is frustrating. Let me investigate that for you.

Primary emotions: neutral, angry, excited, content, sad, and scared.

Laughter

Insert [laughter] to produce a natural laugh.

Oh, I've actually heard that one before! [laughter] Alright, let's get this sorted out.

Tag reference

Malformed tags

System prompt for Amazon Connect AI agents

When Amazon Connect AI Agents generate text that will be spoken, add these speech formatting instructions to your agent's system prompt. Copy and paste the block below directly into your prompt configuration.

Speech output formatting prompt

## Speech Output Formatting
Everything you output will be spoken aloud by text-to-speech.
Follow these rules:

1. GENERAL FORMATTING
- Use full sentences with normal capitalization. Always end with . ? or !
- Use conventional written forms for numbers, dates, acronyms, symbols.
- Do NOT use markdown, bullet points, headers, bold, raw JSON, emoji,
  or special characters.
- Write plain prose only.

2. SPELLING OUT CODES AND IDS
- For codes or IDs that must be read character-by-character, except for phone numbers, wrap in <spell> tags:
  Example: Your confirmation code is <spell>TKT4829XB</spell>.

3. THINGS TO AVOID
- Do not output bullet points, numbered lists, or structured formatting.
- Do not use asterisks, hashtags, or markdown syntax.
- Do not include internal thoughts, reasoning, or meta-commentary.
- Do not use parentheses, brackets, curly braces, or quotation marks.

4. NATURALNESS
- Respond as if speaking directly to the caller.
- Keep responses concise. Avoid long monologues.
- Use natural transitions ("Let me check that for you.",
  "Here is what I found.").

Polyglot add-on

If your bot uses a polyglot voice, add the following to your system prompt to enable multi-language responses:

## Language Handling
You are connected to a polyglot voice that supports: English, German,
Spanish, French, Hindi, Italian, Japanese, Norwegian, Portuguese, and Russian.

Detect the language of the caller input from the transcription. If the
caller speaks in one of the supported locales, respond in that language.
If the caller speaks a language not in the supported list, respond in
English and politely let them know you can assist in English.

Always respond in one language at a time. Do not mix multiple languages
in a single response. If the caller switches languages mid-conversation,
follow their lead and switch your responses to match.

When responding in a non-English language, always use proper accent marks,
diacritics, and locale-appropriate formatting. This ensures the voice
engine produces natural, native-sounding pronunciation.

Contact center examples

IVR greeting

Hi, thanks for calling. How can I help you today?

No tags needed — the engine handles conversational greetings naturally.

Legal disclaimer (slowed)

<speed ratio="0.85"/>This call may be recorded for quality assurance purposes. By continuing, you consent to recording.

Pair with barge-in disabled at the flow level.

Reading back an account number

With spell tags:

Your account number is <spell>1234</spell> <spell>5678</spell> <spell>9012</spell>.

Without spell tags:

Your account number is 1 2 3 4, 5 6 7 8, 9 0 1 2.

Confirmation with reference code

I've created your ticket. Your reference number is <spell>TKT4829XB</spell>. Is there anything else I can help with?

Menu options

Press 1 for billing.<break time="500ms"/>Press 2 for technical support.<break time="500ms"/>Press 3 to speak with an agent.

Spelling a customer's name

Let me confirm — your last name is Smith, spelled <spell>SMITH</spell>. Is that correct?

Common text-to-speech mistakes

Polyglot voice configuration

What are polyglot voices?

Polyglot voices are specially trained to speak multiple languages naturally. Unlike standard voices — which retain the accent of their primary locale — polyglot voices deliver native-sounding pronunciation across their supported languages.

Amazon Connect agentic voice offers the following polyglot voices:

Configuration

Polyglot voices are configured at build time using the Set voice block:

The voice delivers speech in the configured language. To serve multiple languages, create separate contact flows per language, each configured with the appropriate locale for that polyglot voice.

Configuring your AI agent for polyglot responses

When using a polyglot voice, your AI agent receives transcriptions from ASR in whatever language the caller speaks. To respond in the caller's language, add language-handling instructions to your system prompt (see the Polyglot add-on above).

How it works:

Example exchange:

Caller (transcribed): "Necesito verificar el estado de mi pedido."
Agent: "Con gusto le ayudo. ¿Puede proporcionarme su número de pedido?"

English code-switching

If the caller uses English terms (brand names, product codes) within a non-English language, keep your response in the caller's language. Embed English terms naturally — there's no need to translate brand names, codes, or identifiers.

Caller: "Quiero saber el estado de mi Amazon Prime membership."
Agent: "Claro, voy a revisar su Amazon Prime membership. Un momento por favor."

Polyglot best practices

Do:

Don't:

Examples

German conversation with English brand terms:

Ihr Amazon Web Services Konto wurde erfolgreich aktualisiert. Kann ich Ihnen noch mit etwas anderem helfen?

English brand names are preserved as-is within the German response.

Non-polyglot voice speaking another language:

Bonjour, comment puis-je vous aider aujourd'hui?

Any agentic voice can speak another language, but non-polyglot voices carry their primary locale accent. For native-sounding French, use a polyglot voice configured for fr_FR.