Skip to main content

Sambert Speech Synthesis iOS SDK

This document provides a detailed guide on using the Sambert speech synthesis software development kit (SDK) for iOS to convert text into high-quality, expressive speech.

Alibaba Cloud Model Studio has released a workspace-specific domain for the China (Beijing) region: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from https://dashscope.aliyuncs.com to the new domain.{WorkspaceId} is your workspace ID, which can be found on the Workspace Details page in the Alibaba Cloud Model Studio console. The existing domain remains fully functional.
User guide: For more information about the available models and selection suggestions, see Sambert speech synthesis. Online experience: Not supported.

Getting started

  1. Obtain an API key: Obtain an API key
    To grant temporary access permissions to third-party applications or users, or to strictly control access to sensitive data and risky operations such as deletion, use a temporary API key. A temporary API key is valid for 60 seconds and you must obtain a new one after it expires.
  2. Download the SDK and run the sample code:
    • Download the latest SDK package.
    • Decompress the ZIP package and add nuisdk.framework to your project.
    • In Build Phases → Link Binary With Libraries, add nuisdk.framework.
    • In General → Frameworks, Libraries, and Embedded Content, set nuisdk.framework to Embed & Sign.
    • Open the sample project in Xcode. The sample code is in the DashSambertTTSViewController class. You can replace the API key to test the feature.

Call procedure

  1. Initialize the SDK.
  2. Set the required parameters. Use the parameters parameter of the nui_tts_initialize method to set connection and control parameters. Use the nui_tts_set_param method to set speech synthesis effect parameters.
  3. Call nui_tts_play to start speech synthesis.
  4. In the onNuiTtsUserdataCallback callback, you can obtain the audio data and use stream playback. To save the audio locally, you can write the audio to the same file in append mode until the synthesis is complete.
  5. After the task is complete, call nui_tts_release to release the SDK resources.

Request parameters

Connection and control parameters

You can configure the parameters by passing a JSON string in the parameters parameter of the nui_tts_initialize method.
  • Parameter example: The following code provides an example of a JSON string. Not all parameters are listed. You can add parameters as needed during encoding:
{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
    "apikey": "st-****",
    "device_id": "my_device_id"
}
  • Parameter descriptions

    Parameter

    Type

    Required

    Description

    url

    String

    Yes

    The service endpoint. Set to wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference. Replace WorkspaceId with your actual workspace ID.

    apikey

    String

    Yes

    The API key. Use a temporary API key for higher security and a shorter validity period. This reduces the risk of long-term key leakage.

    device_id

    String

    Yes

    A unique string that identifies the end user. You can set this to the in-app user ID or a unique device identifier generated by the client. This ID is mainly used for log tracking and troubleshooting.

    debug_path

    String

    No

    The storage path for log files.

    This parameter takes effect only when save_log is set to YES in the nui_tts_initialize interface call. You must set a log file path. Otherwise, an error is reported.

    A maximum of two log files are retained locally.

    max_log_file_size

    int

    No

    The maximum size of a log file in bytes.

    This parameter takes effect only when save_log is set to YES in the nui_tts_initialize interface call.

    Default value: 104857600 (100 × 1024 × 1024 bytes, which is 100 MiB).

    log_track_level

    int

    No

    The filtering level for log content sent through the log callback (onNuiTtsLogTrackCallback).

    Default value: 2.

    Valid values:

    • 0: LOG_LEVEL_VERBOSE

    • 1: LOG_LEVEL_DEBUG

    • 2: LOG_LEVEL_INFO

    • 3: LOG_LEVEL_WARNING

    • 4: LOG_LEVEL_ERROR

    • 5: LOG_LEVEL_NONE (disables this feature)

    Note: log_track_level and level (set through the nui_tts_initialize interface) together determine the final logs for the callback. A log is sent to the callback only if its level value is greater than or equal to both the log_track_level and level values. For example, if log_track_level is set to 2 (INFO) and level is set to 3 (WARNING), only logs at the WARNING level and higher (value >= 3) are sent to the callback.

Speech synthesis effect parameters

You can set these parameters using the nui_tts_set_param method.

Parameter

Type

Required

Description

model

String

Yes

The speech synthesis model.

format

String

No

The audio coding format. Supported formats are pcm, wav, and mp3.

Default value: pcm.

volume

String

No

The volume.

Default value: 50.

Valid values: [0, 100]. A value of 50 represents the standard volume. The volume has a linear relationship with this value. A value of 0 means mute, and 100 means the maximum volume.

sample_rate

String

No

The sample rate in Hz.

Default value: The default sample rate of the model.

Use the default value of the model. If the sample rate does not match, the server-side performs resampling.

rate

String

No

The speech rate.

Default value: 1.0.

Valid values: [0.5, 2.0]. A value of 1.0 is the standard speech rate. A value less than 1.0 slows down the speech, and a value greater than 1.0 speeds it up.

pitch

String

No

The pitch. This value is a multiplier for pitch adjustment, but its relationship with the perceived pitch change is not strictly linear or logarithmic. Test to find a suitable value.

Default value: 1.0.

Valid values: [0.5, 2.0]. A value of 1.0 is the natural pitch for the timbre. A value greater than 1.0 raises the pitch, and a value less than 1.0 lowers it.

word_timestamp_enabled

String

No

Specifies whether to enable word-level timestamps.

Default value: 0.

Valid values:

  • 1: Enable.

  • 0: Disable.

phoneme_timestamp_enabled

String

No

Specifies whether to enable phoneme-level timestamps. This parameter takes effect only when word_timestamp_enabled is set to 1 (enabled).

Default value: 0.

Valid values:

  • 1: Enable.

  • 0: Disable.

enable_audio_decoder

String

No

Specifies whether to enable the built-in audio decoder.

Default value: 0.

Valid values:

  • 1: Enable. If format is set to mp3, set this parameter to "1" to enable the built-in decoder of the SDK. In this case, onTtsDataCallback returns the decoded PCM data.

  • 0: Disable.

Key interfaces

NeoNuiTts

nui_tts_initialize

Initializes the speech synthesis SDK instance. The SDK is a singleton. Do not initialize the instance again before you call nui_tts_release.
  • Method signature
-(int) nui_tts_initialize:(const char *)parameters
                 logLevel:(NuiSdkLogLevel)level
                  saveLog:(BOOL)save_log;
  • Parameter descriptions

    Parameter

    Type

    Description

    parameters

    char*

    A JSON string that contains authentication, connection, and debugging parameters. For more information, see Connection and control parameters.

    level

    NuiSdkLogLevel

    Controls the logging level of the SDK itself.

    save_log

    BOOL

    Specifies whether to save binary logs. If this parameter is set to YES, you must use the debug_path parameter to specify a path in Connection and control parameters. You can also use max_log_file_size to set the file size.

  • Return value description

nui_tts_set_param

Before calling nui_tts_play, you must set the speech synthesis effect parameters as key-value pairs.
  • Method signature
-(int) nui_tts_set_param:(const char *)param
                   value:(const char *)value;

nui_tts_get_param

Retrieves a parameter value. This method is mainly used for troubleshooting.
  • Method signature
-(const char *) nui_tts_get_param:(const char *)param;
  • Parameter descriptions

    Parameter

    Type

    Description

    param

    char*

    The parameter. Currently, only "error_msg" is supported.

  • Return value description Returns the parameter value.

nui_tts_play

Starts a speech synthesis task.
  • Method signature
-(int) nui_tts_play:(const char *)priority
             taskId:(const char *)taskid
              text:(const char *)text;
  • Parameter descriptions

    Parameter

    Type

    Description

    priority

    char*

    The task priority. Set this parameter to 1.

    taskid

    char*

    The task ID. If you pass null, the SDK automatically generates a task ID.

    text

    char*

    The text to be synthesized.

  • Return value description

nui_tts_pause

Pauses the current speech synthesis task. After the task is paused, you can resume it by calling nui_tts_resume or cancel it by calling nui_tts_cancel. The SDK does not support starting new synthesis tasks while a task is paused. Note: This operation only pauses data from being pulled from the server-side. Audio data that is cached in the player continues to play.
  • Method signature
-(int) nui_tts_pause;
  • Return value description

nui_tts_resume

Resumes a paused speech synthesis task.
  • Method signature
-(int) nui_tts_resume;
  • Return value description

nui_tts_cancel

Cancels a synthesis task. Note: This operation only cancels data from being pulled from the server-side. Audio data that is cached in the player continues to play.
  • Method signature
-(int) nui_tts_cancel:(const char *)taskid;
  • Parameter descriptions

    Parameter

    Type

    Description

    taskid

    char*

    The ID of the task to cancel. If you pass null, all paused or ongoing synthesis tasks are canceled.

  • Return value description

nui_tts_release

Releases all internal SDK resources and forcibly terminates all ongoing synthesis tasks. After this method is called, the SDK instance becomes unavailable. To use the instance again, you must call nui_tts_initialize to re-initialize it.
  • Method signature
-(int) nui_tts_release;
  • Return value description

NeoNuiTtsDelegate: Listener callbacks

onNuiTtsEventCallback: Listen for events

  • Method signature
- (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code;
  • Parameter descriptions

    Parameter

    Type

    Description

    event

    NuiSdkTtsEvent

    The callback event.

    taskid

    char*

    The speech synthesis task ID.

    code

    int

    Valid only when the TTS_EVENT_ERROR event occurs.

onNuiTtsUserdataCallback: Listen for audio data and timestamp information

  • Method signature
- (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id;
  • Parameter descriptions

    Parameter

    Type

    Description

    info

    char*

    The timestamp result in JSON format. This takes effect when the speech synthesis effect parameter word_timestamp_enabled is set to "1".

    info_len

    int

    The data length of the info field. You can ignore this.

    buffer

    char*

    Returns the audio data of the current segment.

    len

    int

    The length of the audio data in bytes.

    task_id

    char*

    The speech synthesis task ID.

onNuiTtsLogTrackCallback: Listen for tracking logs

This callback is used to receive detailed internal logs from the SDK for troubleshooting and debugging.
- (void)onNuiTtsLogTrackCallback:(NuiSdkLogLevel)level
                      logMessage:(const char *)log;

NuiSdkTtsEvent: Event types

EventDescription
TTS_EVENT_STARTThe synthesis task starts. Audio data will be returned soon.
TTS_EVENT_ENDThe synthesis task ends normally. All audio data has been sent through the callback.
TTS_EVENT_CANCELThe synthesis task is canceled.
TTS_EVENT_PAUSEThe synthesis task is paused.
TTS_EVENT_RESUMEThe synthesis task is resumed.
TTS_EVENT_ERRORAn error occurred during synthesis. You can call nui_tts_get_param: "error_msg" to get detailed error information.
{
  "header": {
    "task_id": "xxxxxxxxx",
    "event": "task-failed",
    "error_code": "InvalidParameter",
    "error_message": "Please ensure input text is valid.",
    "attributes": {}
  },
  "payload": {}
}

Model list

The default sample rate is the optimal sample rate for the current model. By default, the output is based on this sample rate. Downsampling and upsampling are also supported. For example, for the Zhimiao voice, the default sample rate is 16 kHz. You can downsample it to 8 kHz, but upsampling it to 48 kHz does not provide additional benefits.
TimbreAudio sample (Right-click to save the audio)model parameterTimestamp supportScenariosFeaturesLanguageDefault sample rate (Hz)
Zhinansambert-zhinan-v1YesGeneral scenariosAdvertising male voiceChinese and English48 kHz
Zhiqisambert-zhiqi-v1YesGeneral scenariosGentle female voiceChinese and English48 kHz
Zhichusambert-zhichu-v1YesNews broadcastingA Bite of China male voiceChinese and English48 kHz
Zhidesambert-zhide-v1YesNews broadcastingNews male voiceChinese and English48 kHz
Zhijiasambert-zhijia-v1YesNews broadcastingStandard female voiceChinese and English48 kHz
Zhirusambert-zhiru-v1YesNews broadcastingNews female voiceChinese and English48 kHz
Zhiqiansambert-zhiqian-v1YesDubbing and news broadcastingInformation female voiceChinese and English48 kHz
Zhixiangsambert-zhixiang-v1YesDubbingMagnetic male voiceChinese and English48 kHz
Zhiweisambert-zhiwei-v1YesReading product introductionsLolita female voiceChinese and English48 kHz
Zhihaosambert-zhihao-v1YesGeneral scenariosConsultation male voiceChinese and English16 KB
Zhijingsambert-zhijing-v1YesGeneral scenariosStrict female voiceChinese and English16 k
Zhimingsambert-zhiming-v1YesGeneral scenariosHumorous male voiceChinese and English16 KB
Zhimosambert-zhimo-v1YesGeneral scenariosEmotional male voiceChinese and English16 kHz
Zhinasambert-zhina-v1YesGeneral scenariosZhejiang Mandarin female voiceChinese and English16 kHz
Zhishusambert-zhishu-v1YesGeneral scenariosInformation male voiceChinese and English16 KB
Zhishasambert-zhistella-v1YesGeneral scenariosIntellectual female voiceChinese and English16 kHz
Zhitingsambert-zhiting-v1YesGeneral scenariosRadio female voiceChinese and English16 kHz
Zhixiaosambert-zhixiao-v1YesGeneral scenariosInformation female voiceChinese and English16 kHz
Zhiyasambert-zhiya-v1YesGeneral scenariosStrict female voiceChinese and English16 kHz
Zhiyesambert-zhiye-v1YesGeneral scenariosYoung male voiceChinese and English16 kHz
Zhiyingsambert-zhiying-v1YesGeneral scenariosCute child voiceChinese and English16 kHz
Zhiyuansambert-zhiyuan-v1YesGeneral scenariosCaring sister voiceChinese and English16 kHz
Zhiyuesambert-zhiyue-v1YesCustomer serviceGentle female voiceChinese and English16 kHz
Zhiguisambert-zhigui-v1YesReading product introductionsLivestreaming female voiceChinese and English16 kHz
Zhishuosambert-zhishuo-v1YesDigital humanNatural male voiceChinese and English16 kHz
Zhimiao (multi-emotional)sambert-zhimiao-emo-v1YesReading product introductions, digital humans, and livestreamingMulti-emotional female voiceChinese and English16 kHz
Zhimaosambert-zhimao-v1YesReading product introductions, dubbing, digital humans, and livestreamingLivestreaming female voiceChinese and English16 kHz
Zhilunsambert-zhilun-v1YesDubbingSuspense narrationChinese and English16 kHz
Zhifeisambert-zhifei-v1YesDubbingPassionate narrationChinese and English16 KB
Zhidasambert-zhida-v1YesNews broadcastingStandard male voiceChinese and English16 KB
Camilasambert-camila-v1NoGeneral scenariosSpanish female voiceSpanish16 kHz
Perlasambert-perla-v1NoGeneral scenariosItalian female voiceItalian16 k
Indahsambert-indah-v1NoGeneral scenariosIndonesian female voiceIndonesian16 kHz
Clarasambert-clara-v1NoGeneral scenariosFrench female voiceFrench16 kHz
Hannasambert-hanna-v1NoGeneral scenariosGerman female voiceGerman16 kHz
Bethsambert-beth-v1YesGeneral scenariosConsultation female voiceAmerican English16 kHz
Bettysambert-betty-v1YesGeneral scenariosCustomer service female voiceAmerican English16 kHz
Callysambert-cally-v1YesGeneral scenariosNatural female voiceAmerican English16 kHz
Cindysambert-cindy-v1YesGeneral scenariosConversational female voiceAmerican English16 kHz
Evasambert-eva-v1YesGeneral scenariosCompanion female voiceAmerican English16 KB
Donnasambert-donna-v1YesGeneral scenariosEducation female voiceAmerican English16 kHz
Briansambert-brian-v1YesGeneral scenariosCustomer service male voiceAmerican English16 kHz
Waansambert-waan-v1NoGeneral scenariosThai female voiceThai16 kHz
Text Generation
Image Generation
Video Generation
World models
3D Model Generation
Audio
  • Audio generation
Realtime API
Text Embedding
Decision Model
TokenPlan
Model Production