API Operation Details

​

Scenarios

• Companion scenarios: Provide personalized companionship by cloning family members' voices. You can use it for smart assistants, in-car navigation, and home entertainment. For example, you can read picture books to family members, control home appliances, or provide educational guidance.
• Education scenarios: Clone teachers' voices to enhance teacher-student interaction. Enrich teaching videos and courseware. This creates a more engaging and vivid learning experience.
• Audio and video industry: Clone streamers' voices for post-production re-recording and dubbing. This improves audio and video production efficiency.
• Intelligent customer service: Clone account managers' voices to provide voice services. This includes customer callbacks and marketing calls, which enhances service personalization and human interaction.

​

Benefits

• Low audio sample requirements: You can clone voices with only 10 to 20 seconds of recording. This significantly reduces recording costs and improves efficiency.
• High fidelity: Using Alibaba Speech Lab's self-developed CosyVoice generative neural network Large Language Model (LLM) algorithm, combined with cutting-edge zero-shot learning technology, it accurately restores real human voices in terms of intonation, rhythm, and emotional expression, making them difficult to distinguish from real recordings.
• Instant synthesis: It restores real voice timbre in seconds, providing an efficient, real-time voice cloning service.

​

Supported Models

​

Notes

• Audio file format requirements:
  • Number of sound channels: Mono or Stereo
  • Sample rate: Greater than or equal to 16,000 Hz
  • Format: WAV (16bit), MP3, M4A
  • File size: Up to 10 MB
  • Audio duration: 10 to 20 seconds. Do not exceed 60 seconds. Maintain continuity when reading. Include at least one continuous speech segment longer than 5 seconds.
• Voice cloning quantity limit: Each Alibaba Cloud account can clone a maximum of 1,000 voices. If your needs exceed this limit, contact our pre-sales team in advance or join the developer group. Voices that remain unused for over 1 year will be taken offline.
• Copyright and legality: You are responsible for the ownership and legal rights to use the provided voice. Read the Terms of Service.
• Using cloned voices: Voices (VoiceName) generated using the voice cloning service are used in the same way as official default voices, such as longxiaoxia, provided in the Java SDK.
• Service invocation method: The voice cloning service currently supports API invocation only.

​

Billing

​

Prerequisites

• You have activated the service and obtained an API key. We recommend that you configure your API key as an environment variable (being phased out and integrated into the Configure API Key procedure) to reduce the risk of API key leakage.
• Install the latest version of the SDK.
• An audio URL accessible over the public network is available. We recommend uploading the audio to OSS.

​

API Details

​

Create Voice

def create_voice(self, target_model: str, prefix: str, url: str) -> str:
    '''
    Create a voice
    param: target_model The model used for voice cloning. Supported models are cosyvoice-v1 and cosyvoice-v2.
    param: prefix A custom prefix for the voice. Only digits and lowercase letters are allowed, with a maximum of 10 characters.
    param: url The URL of the audio file used to clone the voice. This URL must be publicly accessible.
    return: The voice ID
    '''

/**
 * Create a voice
 *
 * @param targetModel The model used for voice cloning. Supported models are cosyvoice-v1 and cosyvoice-v2.
 * @param prefix A custom prefix for the voice. Only digits and lowercase letters are allowed, with a maximum of 10 characters.
 * @param url The URL of the audio file used to clone the voice. This URL must be publicly accessible.
 * @return Voice object
 * @throws NoApiKeyException If the API key is empty
 * @throws InputRequiredException If a required parameter is empty
 */
public Voice createVoice(String targetModel, String prefix, String url) throws NoApiKeyException, InputRequiredException

https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization

Authorization: Bearer {api-key} // Replace with your API key
Content-Type: application/json

{
    "model": "voice-enrollment",
    "input": {
        "action": "create_voice",
        "target_model": "cosyvoice-v2",
        "prefix": "testpfx",
        "url": "https://yourAudioFileUrl"
    }
}

curl -X POST https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "voice-enrollment",
    "input": {
        "action": "create_voice",
        "target_model": "cosyvoice-v1",
        "prefix": "testpfx",
        "url": "https://yourAudioFileUrl"
    }
}'

{
    "output": {
        "voice_id": "yourVoiceId"
    },
    "usage": {
        "count": 1
    },
    "request_id": "yourRequestId"
}

​

Query all voices

def list_voices(self, prefix=None, page_index: int = 0, page_size: int = 10) -> List[dict]:
    '''
    Queries all created voices.
    param: prefix The voice prefix. If set to None, all voices are returned.
    param: page_index The page index for the query.
    param: page_size The number of entries per page.
    return: List[dict] A list of voices. Each entry contains the voice ID, creation time, modification time, and status.
    '''

[
    {
        "gmt_create": "2024-09-13 11:29:41",
        "voice_id": "yourVoiceId",
        "gmt_modified": "2024-09-13 11:29:41",
        "status": "OK"
    },
    {
        "gmt_create": "2024-09-13 13:22:38",
        "voice_id": "yourVoiceId",
        "gmt_modified": "2024-09-13 13:22:38",
        "status": "OK"
    }
]

/**
 * Queries all created voices. The default page index is 0 and the default page size is 10.
 *
 * @param prefix The custom voice prefix. It can contain only digits and lowercase letters and must be less than 10 characters long. This parameter can be null.
 * @return Voice[] An array of voice objects.
 * @throws NoApiKeyException If the API key is empty.
 * @throws InputRequiredException If a required parameter is empty.
 */
public Voice[] listVoice(String prefix) throws NoApiKeyException, InputRequiredException

/**
 * Queries all created voices.
 *
 * @param prefix The custom voice prefix. It can contain only digits and lowercase letters and must be less than 10 characters long.
 * @param pageIndex The page index for the query.
 * @param pageSize The number of entries per page.
 * @return Voice[] An array of voice objects.
 * @throws NoApiKeyException If the API key is empty.
 * @throws InputRequiredException If a required parameter is empty.
 */
public Voice[] listVoice(String prefix, int pageIndex, int pageSize) throws NoApiKeyException, InputRequiredException

[
    {
        "gmt_create": "2024-09-13 11:29:41",
        "voice_id": "yourVoiceId",
        "gmt_modified": "2024-09-13 11:29:41",
        "status": "OK"
    },
    {
        "gmt_create": "2024-09-13 13:22:38",
        "voice_id": "yourVoiceId",
        "gmt_modified": "2024-09-13 13:22:38",
        "status": "OK"
    }
]

https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization

Authorization: Bearer {api-key} // Replace {api-key} with your own API key.
Content-Type: application/json

{
    "model": "voice-enrollment",
    "input": {
        "action": "list_voice",
        "prefix": "testpfx",
        "page_index": 0,
        "page_size": 10
    }
}

curl -X POST https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "voice-enrollment",
    "input": {
        "action": "list_voice",
        "prefix": "testpfx",
        "page_index": 0,
        "page_size": 10
    }
}'

{
    "output": {
        "voice_list": [
            {
                "gmt_create": "2024-12-11 13:38:02",
                "voice_id": "yourVoiceId",
                "gmt_modified": "2024-12-11 13:38:02",
                "status": "OK"
            }
        ]
    },
    "usage": {
        "count": 1
    },
    "request_id": "yourRequestId"
}

​

Query a specific voice

def query_voices(self, voice_id: str) -> List[str]:
    '''
    Query all created voices.
    param: voice_id: voice to query
    return: bytes: audio used to register the voice
    '''

{
    "gmt_create": "2024-09-13 11:29:41",
    "resource_link": "https://yourAudioFileUrl",
    "target_model": "cosyvoice-v1",
    "gmt_modified": "2024-09-13 11:29:41",
    "status": "OK"
}

/**
 * Queries a specific voice.
 *
 * @param voiceId The ID of the voice to query.
 * @return Voice A Voice object that contains the status information and the URL of the audio file for cloning.
 * @throws NoApiKeyException If the API key is empty.
 * @throws InputRequiredException If a required parameter is empty.
 */
public Voice queryVoice(String voiceId) throws NoApiKeyException, InputRequiredException

{
    "gmt_create": "2024-09-13 11:29:41",
    "resource_link": "https://yourAudioFileUrl",
    "target_model": "cosyvoice-v1",
    "gmt_modified": "2024-09-13 11:29:41",
    "status": "OK"
}

https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization

Authorization: Bearer {api-key} // Replace {api-key} with your API key.
Content-Type: application/json

{
    "model": "voice-enrollment",
    "input": {
        "action": "query_voice",
        "voice_id": "yourVoiceId"
    }
}

curl -X POST https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "voice-enrollment",
    "input": {
        "action": "query_voice",
        "voice_id": "yourVoiceId"
    }
}'

{
    "output": {
        "gmt_create": "2024-12-11 13:38:02",
        "resource_link": "https://yourAudioFileUrl",
        "target_model": "cosyvoice-v1",
        "gmt_modified": "2024-12-11 13:38:02",
        "status": "OK"
    },
    "usage": {
        "count": 1
    },
    "request_id": "2450f969-d9ea-9483-bafc-************"
}

​

Update voice

def update_voice(self, voice_id: str, url: str) -> None:
    '''
    Update a voice.
    param: voice_id The voice ID.
    param: url The URL of the audio file for cloning.
    '''

/**
 * Update a voice.
 *
 * @param voiceId The voice to update.
 * @param url The URL of the audio file for cloning.
 * @throws NoApiKeyException if the API key is empty.
 * @throws InputRequiredException if a required parameter is empty.
 */
public void updateVoice(String voiceId, String url)
    throws NoApiKeyException, InputRequiredException

https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization

Authorization: Bearer {api-key} // Replace {api-key} with your own API key.
Content-Type: application/json

{
    "model": "voice-enrollment",
    "input": {
        "action": "update_voice",
        "voice_id": "yourVoiceId",
        "url": "https://yourAudioFileUrl"
    }
}

curl -X POST https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "voice-enrollment",
    "input": {
        "action": "update_voice",
        "voice_id": "yourVoiceId",
        "url": "https://yourAudioFileUrl"
    }
}'

{
    "output": {},
    "usage": {
        "count": 1
    },
    "request_id": "yourRequestId"
}

​

Delete a voice

def delete_voice(self, voice_id: str) -> None:
    '''
    Delete a voice.

    Args:
        voice_id: The ID of the voice to delete.
    '''

/**
 * Delete a voice.
 *
 * @param voiceId The ID of the voice to delete.
 * @throws NoApiKeyException If the API key is empty.
 * @throws InputRequiredException If a required parameter is empty.
 */
public void deleteVoice(String voiceId) throws NoApiKeyException, InputRequiredException

https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization

Authorization: Bearer {api-key} // Replace {api-key} with your own API key.
Content-Type: application/json

{
    "model": "voice-enrollment",
    "input": {
        "action": "delete_voice",
        "voice_id": "yourVoiceId"
    }
}

curl -X POST https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "voice-enrollment",
    "input": {
        "action": "delete_voice",
        "voice_id": "yourVoiceId"
    }
}'

{
    "output": {},
    "usage": {
        "count": 1
    },
    "request_id": "yourRequestId"
}

​

Use a cloned voice for speech synthesis

1. Obtain the voice ID Call the Create Voice or Query Voice List API to obtain the voice ID.
2. Call the CosyVoice speech synthesis API Select the API documentation for your programming language. Pass the voice ID that you obtained in Step 1 as the voice parameter to the synthesis API:
   • Java: Java SDK
   • Python: Python SDK
   • Other programming languages (such as Go, C#, PHP, and Node.js): WebSocket API

import os
import dashscope
from dashscope.audio.tts_v2 import VoiceEnrollmentService, SpeechSynthesizer

dashscope.api_key = os.getenv('DASHSCOPE_API_KEY')  # If you have not configured the environment variable, provide your API key here.
url = "https://your-audio-file-url"  # Replace this with the actual URL.
prefix = 'prefix'
target_model = "cosyvoice-v2"

# Create a service instance for voice enrollment.
service = VoiceEnrollmentService()

# Call the create_voice method to clone the voice and generate a voice_id.
voice_id = service.create_voice(target_model=target_model, prefix=prefix, url=url)
print("requestId: ", service.get_last_request_id())
print(f"your voice id is {voice_id}")

# Use the cloned voice for speech synthesis.
synthesizer = SpeechSynthesizer(model=target_model, voice=voice_id)
audio = synthesizer.call("How is the weather today?")
print("requestId: ", synthesizer.get_last_request_id())

# Save the synthesized audio to a local file.
with open("output.mp3", "wb") as f:
    f.write(audio)

​

Error codes

class VoiceEnrollmentException(Exception):
  def __init__(self, status_code: int, code: str, error_message: str)