Skip to main content

Sambert speech synthesis Python SDK

This topic describes the parameters and interfaces of the Sambert speech synthesis Python SDK.

Alibaba Cloud Model Studio has released a workspace-specific domain for the China (Beijing) region. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.Replace {WorkspaceId} with your actual Workspace ID. The existing domain remains fully functional.
User guide: For more information about model introductions and selection suggestions, see Real-time Speech Synthesis - CosyVoice/Sambert. Online experience: Not supported.

Prerequisites

You have activated the service and obtained and configured an API key. To prevent security risks from code leaks, set the API key as an environment variable instead of hard-coding it in your code.

Getting started

The SpeechSynthesizer class provides interfaces for synchronous and streaming calls. Choose the appropriate method based on your scenario:
  • Synchronous call: Submit text in a single request. The server then processes the text and returns the complete synthesized audio. This is a blocking process, which means the client must wait for the server to finish before it can proceed. This method is suitable for short text synthesis.
  • Streaming call: Send text to the server in a single request. The synthesized audio is then streamed back in real time. You cannot send the text in segments. This method is suitable for scenarios that require low latency.

Non-streaming call

Submit a single speech synthesis task and retrieve the complete result at once. You do not need to use a callback because intermediate results are not streamed.
image
To make a synchronous call, use the call method of the SpeechSynthesizer class. You can use the call method to set request parameters. Ensure that you do not set the callback parameter. After the task is complete, the method returns the audio data and timestamp information (SpeechSynthesisResult).
The following example shows how to use the synchronous interface to call the Zhichu (sambert-zhichu-v1) model to synthesize the text "How is the weather today?". The synthesized audio is in the WAV format with a sample rate of 48 kHz and is saved to a file named output.wav.
# coding=utf-8
import sys
import dashscope
from dashscope.audio.tts import SpeechSynthesizer
# If you do not configure the API key as an environment variable, replace apiKey with your API key.
# dashscope.api_key = "apiKey"
# Replace "{WorkspaceId}" with your workspace ID.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'
result = SpeechSynthesizer.call(model='sambert-zhichu-v1',
                                text='How is the weather today?',
                                sample_rate=48000,
                                format='wav')
if result.get_audio_data() is not None:
    with open('output.wav', 'wb') as f:
        f.write(result.get_audio_data())
    print('SUCCESS: get audio data: %dbytes in output.wav' %
          (sys.getsizeof(result.get_audio_data())))
else:
    print('ERROR: response is %s' % (result.get_response()))

Streaming call

Submit a single speech synthesis task. The results are streamed back through a callback method in the ResultCallback class.
image
  1. Instantiate ResultCallback.
  2. Call the call method of the SpeechSynthesizer class to synthesize speech. You can use the call method to set request parameters. Ensure that you set the callback parameter.
The following example shows how to use the streaming interface to call the Zhichu (sambert-zhichu-v1) model. It synthesizes the text "How is the weather today?" into streaming audio in the default WAV format with a 48 kHz sample rate, and retrieves the corresponding timestamp.
# coding=utf-8

import sys
import dashscope
from dashscope.api_entities.dashscope_response import SpeechSynthesisResponse
from dashscope.audio.tts import ResultCallback, SpeechSynthesizer, SpeechSynthesisResult

# If you do not configure the API key as an environment variable, replace apiKey with your API key.
# dashscope.api_key = "apiKey"
# Replace "{WorkspaceId}" with your workspace ID.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'

class Callback(ResultCallback):
    def on_open(self):
        print('Speech synthesizer is opened.')

    def on_complete(self):
        print('Speech synthesizer is completed.')

    def on_error(self, response: SpeechSynthesisResponse):
        print('Speech synthesizer failed, response is %s' % (str(response)))

    def on_close(self):
        print('Speech synthesizer is closed.')

    def on_event(self, result: SpeechSynthesisResult):
        if result.get_audio_frame() is not None:
            print('audio result length:', sys.getsizeof(result.get_audio_frame()))

        if result.get_timestamp() is not None:
            print('timestamp result:', str(result.get_timestamp()))

callback = Callback()
SpeechSynthesizer.call(model='sambert-zhichu-v1',
                       text='How is the weather today?',
                       sample_rate=48000,
                       callback=callback,
                       word_timestamp_enabled=True,
                       phoneme_timestamp_enabled=True)

Request parameters

You can set the request parameters in the call method of the SpeechSynthesizer class.
ParameterTypeDefault valueRequiredDescription
modelstr
YesThe name of the voice model to use for speech synthesis. For a complete list, see Model list.
textstr
YesThe text to be synthesized. The text must be UTF-8 encoded and cannot be empty.Maximum number of characters: 10,000.
Character calculation rule: One Chinese character, one English letter, one punctuation mark, or one space between sentences is counted as one character.
The SSML format is supported. For more information, see Introduction to SSML.
formatstrwavNoThe encoding format of the synthesized audio. The pcm, wav, and mp3 formats are supported.
sample_rateint16000NoThe sample rate of the synthesized audio in Hz. We recommend that you use the model's default sample rate (see the Model list). If there is a mismatch, the service performs the necessary upsampling or downsampling.
volumeint50NoThe volume of the synthesized audio. Valid values: 0 to 100.
ratefloat1.0NoThe speech rate of the synthesized audio. Valid values: 0.5 to 2.
  • 0.5: indicates 0.5 times the default speech rate.
  • 1: indicates the default speech rate. The default speech rate is the synthesis rate of the default output of the model. The speech rate may vary slightly depending on the speaker. The rate is about four characters per second.
  • 2: indicates twice the default speech rate.
pitchfloat1.0NoThe pitch of the synthesized audio. Valid values: 0.5 to 2.
word_timestamp_enabledboolFalseNoSpecifies whether to enable word-level timestamps. The default value is false.
phoneme_timestamp_enabledboolFalseNoSpecifies whether to generate phoneme-level timestamp information based on word-level timestamps (word_timestamp_enabled). The default value is false.
callbackResultCallback
NoIf you set the callback parameter, the streaming call mode is used.If the callback parameter is not set, the invocation runs in non-streaming mode.For the callback implementation, see Callback interface (ResultCallback).

Key interfaces

SpeechSynthesizer class

You can import the SpeechSynthesizer class with from dashscope.audio.tts import SpeechSynthesizer. The key methods are as follows:
Interface/MethodParametersReturn valueDescription
@classmethod
def call(cls,
         model: str,
         text: str,
         callback: ResultCallback = None,
         workspace: str = None,
         **kwargs) -> SpeechSynthesisResult:
The synthesis result is SpeechSynthesisResult. You must handle this result for non-streaming invocations, but not for asynchronous invocations.Starts a speech synthesis task. The behavior depends on whether you pass the callback parameter:
  • If you do not pass the callback parameter, the call function returns all speech synthesis results after the synthesis is complete.
  • If you pass the callback parameter, the server calls the corresponding function in callback to stream the speech synthesis results during the synthesis process.
You can use the call method to set the request parameters.

Callback interface (ResultCallback)

When you make a streaming call, the server returns key information and data to the client through a callback. You must implement a callback method to process the information and data that is returned by the server. 
class Callback(ResultCallback):
    def on_open(self):
        print('Successfully connected to the server')

    def on_complete(self):
        print('Task completed')

    def on_error(self, response: SpeechSynthesisResponse):
        print('Error: %s' % (str(response)))

    def on_close(self):
        print('The connection to the server is closed.')

    def on_event(self, result: SpeechSynthesisResult):
        if result.get_audio_frame() is not None:
            print('Received binary audio data:', result.get_audio_frame())

        if result.get_timestamp() is not None:
            print('Received timestamp data:', str(result.get_timestamp()))

callback = Callback()
Interface/MethodParametersReturn valueDescription
def on_open(self) -> None
NoneNoneThis method is called immediately after a connection is established with the service.
def on_event(self, result: SpeechSynthesisResult) -> None
result: Audio data and timestamp information (SpeechSynthesisResult)NoneThis method is called when the server returns synthetic data.
def on_complete(self) -> None
NoneNoneThis method is called after all synthetic data is returned.
def on_error(self, response: SpeechSynthesisResponse)
response: Exception informationNoneThis method is called when an exception occurs during the call or the service returns an error.
def on_close(self) -> None
NoneNoneThis method is called after the service closes the connection.

Response results

Audio data and timestamp information (SpeechSynthesisResult)

SpeechSynthesisResult encapsulates the speech synthesis result. The commonly used methods are get_audio_frame, get_timestamp, get_audio_data, and get_timestamps.
Interface/MethodParametersReturn valueDescription
def get_audio_frame(self) -> bytes
NoneThe current synthesized binary audio data segmentIn streaming synthesis, obtains the current synthesized audio frame data.
This function must be used in the event callback method during streaming synthesis.
def get_timestamp(self) -> Dict[str, str]
NoneThe currently synthesized sentence's timestamp informationIn streaming synthesis, obtains the timestamp information corresponding to the currently synthesized sentence.
This function must be used in the event callback method during streaming synthesis.
def get_audio_data(self) -> bytes
NoneThe complete binary audio dataObtains the complete binary audio data.
def get_timestamps(self) -> List[Dict[str, str]]
NoneSentence-level timestamp informationObtains the timestamp information corresponding to all sentences.

Timestamp information

The get_timestamp method of SpeechSynthesisResult retrieves the timestamp information for the currently synthesized sentence. The get_timestamps method retrieves the timestamp information for all sentences. The following code provides an example of the timestamp information for a single sentence. words corresponds to word-level timestamp information, and phonemes corresponds to phoneme-level timestamp information: 
{
    "begin_time":0,
    "end_time":1412,
    "words":[
        {
            "text":"jin",
            "begin_time":0,
            "end_time":200,
            "phonemes":[
                {
                    "begin_time":0,
                    "end_time":82,
                    "text":"j_c",
                    "tone":1
                },
                {
                    "begin_time":82,
                    "end_time":200,
                    "text":"in_c",
                    "tone":1
                }
            ]
        }
    ]
}
The parameters are as follows:

Parameter

Type

Description

begin_time

int

The start time of a sentence, word, or phoneme in milliseconds.

end_time

int

The end time of a sentence, word, or phoneme in milliseconds.

words

list

The word-level timestamp information. The word_timestamp_enabled parameter in the request must also be set to true.

text

str

The text information.

phonemes

list

The phoneme-level timestamp information. The phoneme_timestamp_enabled parameter in the request must also be set to true.

tone

str

The tone.

  • In English, 0, 1, and 2 represent unstressed, primary stress, and secondary stress, respectively.

  • In Pinyin, 1, 2, 3, 4, and 5 represent the first, second, third, fourth, and neutral tones, respectively.

Error codes

If an API call fails and returns an error message, see Error codes to resolve the issue.

More examples

For more examples, see GitHub.

FAQ

For more information, see the QA on GitHub.

Model list

The default sample rate is the optimal sample rate for the current model. By default, the output is based on this sample rate. Downsampling and upsampling are also supported. For example, for the Zhimiao voice, the default sample rate is 16 kHz. You can downsample it to 8 kHz, but upsampling it to 48 kHz does not provide additional benefits.
TimbreAudio sample (Right-click to save the audio)model parameterTimestamp supportScenariosFeaturesLanguageDefault sample rate (Hz)
Zhinansambert-zhinan-v1YesGeneral scenariosAdvertising male voiceChinese and English48 kHz
Zhiqisambert-zhiqi-v1YesGeneral scenariosGentle female voiceChinese and English48 kHz
Zhichusambert-zhichu-v1YesNews broadcastingA Bite of China male voiceChinese and English48 kHz
Zhidesambert-zhide-v1YesNews broadcastingNews male voiceChinese and English48 kHz
Zhijiasambert-zhijia-v1YesNews broadcastingStandard female voiceChinese and English48 kHz
Zhirusambert-zhiru-v1YesNews broadcastingNews female voiceChinese and English48 kHz
Zhiqiansambert-zhiqian-v1YesDubbing and news broadcastingInformation female voiceChinese and English48 kHz
Zhixiangsambert-zhixiang-v1YesDubbingMagnetic male voiceChinese and English48 kHz
Zhiweisambert-zhiwei-v1YesReading product introductionsLolita female voiceChinese and English48 kHz
Zhihaosambert-zhihao-v1YesGeneral scenariosConsultation male voiceChinese and English16 KB
Zhijingsambert-zhijing-v1YesGeneral scenariosStrict female voiceChinese and English16 k
Zhimingsambert-zhiming-v1YesGeneral scenariosHumorous male voiceChinese and English16 KB
Zhimosambert-zhimo-v1YesGeneral scenariosEmotional male voiceChinese and English16 kHz
Zhinasambert-zhina-v1YesGeneral scenariosZhejiang Mandarin female voiceChinese and English16 kHz
Zhishusambert-zhishu-v1YesGeneral scenariosInformation male voiceChinese and English16 KB
Zhishasambert-zhistella-v1YesGeneral scenariosIntellectual female voiceChinese and English16 kHz
Zhitingsambert-zhiting-v1YesGeneral scenariosRadio female voiceChinese and English16 kHz
Zhixiaosambert-zhixiao-v1YesGeneral scenariosInformation female voiceChinese and English16 kHz
Zhiyasambert-zhiya-v1YesGeneral scenariosStrict female voiceChinese and English16 kHz
Zhiyesambert-zhiye-v1YesGeneral scenariosYoung male voiceChinese and English16 kHz
Zhiyingsambert-zhiying-v1YesGeneral scenariosCute child voiceChinese and English16 kHz
Zhiyuansambert-zhiyuan-v1YesGeneral scenariosCaring sister voiceChinese and English16 kHz
Zhiyuesambert-zhiyue-v1YesCustomer serviceGentle female voiceChinese and English16 kHz
Zhiguisambert-zhigui-v1YesReading product introductionsLivestreaming female voiceChinese and English16 kHz
Zhishuosambert-zhishuo-v1YesDigital humanNatural male voiceChinese and English16 kHz
Zhimiao (multi-emotional)sambert-zhimiao-emo-v1YesReading product introductions, digital humans, and livestreamingMulti-emotional female voiceChinese and English16 kHz
Zhimaosambert-zhimao-v1YesReading product introductions, dubbing, digital humans, and livestreamingLivestreaming female voiceChinese and English16 kHz
Zhilunsambert-zhilun-v1YesDubbingSuspense narrationChinese and English16 kHz
Zhifeisambert-zhifei-v1YesDubbingPassionate narrationChinese and English16 KB
Zhidasambert-zhida-v1YesNews broadcastingStandard male voiceChinese and English16 KB
Camilasambert-camila-v1NoGeneral scenariosSpanish female voiceSpanish16 kHz
Perlasambert-perla-v1NoGeneral scenariosItalian female voiceItalian16 k
Indahsambert-indah-v1NoGeneral scenariosIndonesian female voiceIndonesian16 kHz
Clarasambert-clara-v1NoGeneral scenariosFrench female voiceFrench16 kHz
Hannasambert-hanna-v1NoGeneral scenariosGerman female voiceGerman16 kHz
Bethsambert-beth-v1YesGeneral scenariosConsultation female voiceAmerican English16 kHz
Bettysambert-betty-v1YesGeneral scenariosCustomer service female voiceAmerican English16 kHz
Callysambert-cally-v1YesGeneral scenariosNatural female voiceAmerican English16 kHz
Cindysambert-cindy-v1YesGeneral scenariosConversational female voiceAmerican English16 kHz
Evasambert-eva-v1YesGeneral scenariosCompanion female voiceAmerican English16 KB
Donnasambert-donna-v1YesGeneral scenariosEducation female voiceAmerican English16 kHz
Briansambert-brian-v1YesGeneral scenariosCustomer service male voiceAmerican English16 kHz
Waansambert-waan-v1NoGeneral scenariosThai female voiceThai16 kHz
API Usage
Text Generation
Image Generation
Video Generation
World models
3D Model Generation
Audio
  • Audio generation
Real-time Multimodal
Realtime API
Text Embedding
Translation models
Decision Model
More Models
Toolkit/Framework
TokenPlan
Model Production
File Management
Billing
Model Management
More
Sambert speech synthesis Python SDK - 阿里云百炼