Pronunciation

Overview

pronunciations is an optional list on a text-to-speech request that tells the model how to say particular words: a person’s name, a brand, a product term, an acronym. Each entry names a word or phrase as it appears in your input, and either the phones to say (ipa) or the text to say in its place (alias). The model applies an entry everywhere its text appears, so you write the list once and leave your input as it is.

Use it when:

• a name has a reading the model cannot guess, such as a Japanese surname or a rare Chinese character
• a brand or product name must always sound the same
• an acronym should be read as a word (SaaS as “sass”), or a word in one language should be read the way speakers of another say it

Simba 3.5 (simba-3.5) applies pronunciations in every language it speaks except English, which is not supported yet. GET /v1/audio/models lists the languages each model takes them in as pronunciation_languages, so check it when your application chooses a model or a language at runtime. A request whose model or language is not listed refuses the field with 400 pronunciations_unsupported rather than ignoring it.

There is nothing to upload or manage: keep your list in your application and send it with each request that needs it.

A first request

This Spanish request says “Speechify” with the phones given, only where it is capitalized that way, and reads “SaaS” as “sas”:

import base64
import os
from speechify import Speechify
client = Speechify(token=os.environ["SPEECHIFY_API_KEY"])
response = client.audio.speech(
    input="Speechify lee en voz alta tus paneles de SaaS.",
    voice_id="aitana",
    model="simba-3.5",
    language="es-MX",
    audio_format="mp3",
    pronunciations=[
        {"text": "Speechify", "ipa": "s p ˈiː tʃ ɪ f aɪ", "case_sensitive": True},
        {"text": "SaaS", "alias": "sas"},
    ],
)
with open("speech.mp3", "wb") as f:
    f.write(base64.b64decode(response.audio_data))

The streaming endpoints take the same field in the same place.

On the WebSocket input stream

The WebSocket input stream takes the same list in a session.pronunciations message, sent once, before your first input.text:

{"type":"session.pronunciations","pronunciations":[{"text":"SaaS","alias":"sass"}]}

The entries apply to every segment the session speaks. A second message, one sent after text, a model or language without pronunciations, or an invalid entry ends the session with a speech.error carrying the same error codes as the HTTP endpoints, before anything is spoken.

Entries

An entry takes no other fields.

ipa or alias

Use alias when ordinary spelling can say it. An alias is plain text, so it is quick to write and needs no phonetics: "SaaS" as "sass", "SQL" as "sequel", or an English term spelled the way Japanese speakers say it.

Use ipa when spelling cannot say it: a name no spelling pins down, stress on a particular syllable, or a sound the language of your text has no letters for. One phone set covers every language the model speaks. An ipa entry gives the phones of one word, so its text has no spaces: for a name of several words, give one entry per word, or an alias.

Writing phones

• Separate phones with spaces or |. s p ˈiː tʃ ɪ f aɪ is seven phones. Every phone must appear in the phone set exactly as listed, or the request is refused.
• Write a stress mark together with its vowel, as one phone: ˈiː, not ˈ iː. If the stressed vowel you need is not listed, put the stress on its first part: ˈɔ ɪ for a stressed ɔɪ.
• Write a Mandarin tone as a digit after the syllable, as its own phone: ʈʂ ɤ 2 is zhé. 1 to 4 are the four tones and 5 is the neutral tone.
• Keep phones to a minority of the words you send. The model was trained with at most about 20 to 25 percent of words written as phones, and stays reliable under 20 percent. Give phones for the names and terms it gets wrong, not for whole sentences.

IPA from a dictionary or another speech engine often spells some phones differently, so convert it first:

If a phone is refused, split it into the longest pieces the phone set lists: ˈɔɪ becomes ˈɔ ɪ.

How entries match

• Capitalization is ignored unless the entry sets case_sensitive: true. With it, Speechify matches Speechify and not speechify.
• In text written with spaces, an entry matches whole words. Speechify does not match the start of Speechifying.
• An apostrophe inside a word keeps it one word, as in a contraction or a name: don does not match in don't, nor Brien in O'Brien. A possessive 's ends a word, so Amazon matches in Amazon's, and a French or Italian elision starts one, so iPhone and Amazon match in l'iPhone and dell'Amazon.
• In Chinese, Japanese, Korean, Thai, Lao, Khmer and Burmese script, an entry matches anywhere in the text, because words there are not separated by spaces, or in Korean carry their particles attached: 서울 matches in 서울에.
• The longest match wins. With entries for both New York and York, the text New York takes the New York entry.
• Text in decomposed Unicode matches as if it were composed, so an entry for José matches a José written as e and a combining accent, as some systems produce.
• Your entry overrides the model’s own pronunciation, including the readings it chooses automatically for Chinese and Japanese. An ipa entry that gives the reading the model would choose anyway changes nothing.
• An alias replaces the text before the text is normalized, the step that turns numbers, symbols and abbreviations into words. Match text as you wrote it in input, and write an alias the way you would write input.
• An ipa entry matches the text as you wrote it, and its phones follow that word through normalization, so they apply even where normalization respells the word. A word that normalization joins to its neighbours or splits into several, such as $5 (“five dollars”) or McDonald's, takes no phones: give it an alias instead.
• To have a symbol or an emoji read aloud, give it an alias. With {"text": "👍", "alias": "thumbs up"} the model says “thumbs up”; without an alias, a symbol the model does not read is skipped.
• Two entries for the same text are refused, including two that differ only in capitalization, unless both set case_sensitive: true.

Examples

Each example shows the fields to change in a request such as the first request; keep your own voice_id.

Japanese: a surname and a loanword

Japanese names often have several possible readings, so give the one the person uses. Here the surname 柄木田 is read Karakida, and Kubernetes is said the way Japanese speakers say it:

{
  "input": "柄木田さんが Kubernetes のクラスタを設定しました。",
  "model": "simba-3.5",
  "language": "ja-JP",
  "pronunciations": [
    {"text": "柄木田", "ipa": "k a ɾ a k i d a"},
    {"text": "Kubernetes", "alias": "クバネティス"}
  ]
}

One request mixes the two kinds: phones for the name, and an alias in katakana for the loanword.

Mandarin: a rare character in a name

喆 is a rare character used in given names. Read zhé, it is the initial ʈʂ, the final ɤ and the second tone, 2:

{
  "input": "张喆是我们的产品经理。",
  "model": "simba-3.5",
  "language": "zh-CN",
  "pronunciations": [
    {"text": "喆", "ipa": "ʈʂ ɤ 2"}
  ]
}

Pronunciations and SSML

SSML’s <sub alias> also changes what is said, and it works on every model. The two differ in reach:

Use <sub alias> for a one-off substitution, on a model without pronunciations, or when the same word should be read differently in different places. Use pronunciations for names and terms that recur, for a list you maintain once and send with every request, and for anything spelling cannot express.

Speech marks, billing and content policy

• Speech marks still point at your text. start, end and value refer to your input as on any request, never to an alias or the phones. See Speech marks.
• Pronunciations are not billed. A request is billed for the characters of input exactly as without them; an alias longer than its text adds nothing.
• An alias is screened by the content policy together with your input, because it is spoken.
• Entries are recorded with your input text, under the same retention, so zero data retention withholds them too.

Limits

A character is one Unicode code point, as in the character limits. Where SSML splits your input into parts, at a <break> for example, the 20,000 characters apply to each part.

Errors

Both errors return before anything is synthesized or billed.

400 pronunciations_unsupported: the model does not apply pronunciations, or not in the request’s language. A request that omits model gets the default model, and its language is its language, else its voice’s, else English. Simba 3.5 refuses pronunciations in English for now. Send the request without pronunciations, or in a language the model’s pronunciation_languages lists.

400 validation_failed: an entry is malformed, uses a phone outside the phone set, gives phones for more than one word, repeats another entry’s text, the list has more than 200 entries, or the input with its aliases in place is longer than 20,000 characters. fields names every entry and field that is wrong, such as pronunciations[3].ipa. The message describes the first of them and, for a phone, quotes it, adding the phone you probably meant when one of the spellings in Writing phones fixes it. A first entry with "ipa": "s p ˈiː t̠ʃ ɪ f aɪ" returns:

{
  "error": {
    "code": "validation_failed",
    "message": "pronunciations[0].ipa: phone \"t̠ʃ\" is not in this model's phone set; did you mean \"tʃ\"?",
    "fields": {
      "pronunciations[0].ipa": "phone 4 is not in this model's phone set"
    }
  },
  "request_id": "..."
}

fields counts the phone’s position instead of quoting it, so it never repeats what you sent. A suggestion is never applied for you: correct the entry yourself. Fix the entry and send the request again; retrying it unchanged fails the same way. An empty pronunciations list is the same as none, on every model.

The phone set

Simba 3.5 reads the 458 phones below. Phones are compared after Unicode NFC normalization, so a precomposed ẽ and an e followed by a combining tilde are the same phone. Every phone is checked before synthesis: a phone outside the set is a 400, never left out of the audio without a word.

Vowels

Stressed vowels

Consonants

Every lowercase ASCII letter except g is in the set. For the sound in “go”, write the IPA ɡ (U+0261, Latin small letter script g); the keyboard g is refused.

Affricates

Tones and digits

Modifiers and diacritics

A modifier or a diacritic in this table is also a phone on its own. A letter with a diacritic is one phone only where the tables above list the combination; otherwise write the letter and the diacritic as two phones.