The PixVerse - Video Lip Sync model takes a video and audio (or TTS text) as input and generates a lip-synced video aligned with the audio.
Activate the service
- Go to the Alibaba Cloud Model Studio console, search for PixVerse, find the PixVerse model card, and click Activate Now.
- In the pop-up window, confirm the activation and authorization.
Applicable scope
To ensure a successful API call, make sure the model, Endpoint URL, and API Key all belong to the same region. Cross-region calls will fail.
- Select model: Confirm the region to which the model belongs.
- Select URL: Select the Endpoint URL for the corresponding region.
- Configure API Key: Select the region and Obtain an API key, then Configure API key as an environment variable.
The video lip sync examples in this topic apply to the China (Beijing) region.
HTTP call
Since video lip sync tasks take a long time (typically 1-5 minutes), the API uses asynchronous calls. The entire process includes two core steps: "Create task -> Poll for result", as follows:
Step 1: Create a task and obtain the task ID
Beijing region: POST https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis
- After the task is created, use the returned
task_idto query the result. Thetask_idis valid for 24 hours. Do not create duplicate tasks. Instead, use polling to retrieve the result. - For guidance for beginners, see Call APIs with Postman or cURL.
Request parametersHeadersContent-Typestring (Required)The content type of the request. Must be application/json.Authorization string (Required)Authenticates the request with a Model Studio API key. Example: Bearer sk-xxxx.X-DashScope-Async string (Required)Enables asynchronous processing. HTTP requests support only asynchronous calls. Must be enable.Request Bodymodelstring (Required)The model name. Fixed value: pixverse/pixverse-lipsync.input object (Required)The basic input information, including the face video and audio source.
Properties media array (Required)List of media resources. Must contain one video_url type video, and an audio source (choose one):
Element properties type string (Required)Media type identifier. Valid values:
string (Conditionally required)Required when type is video_url or audio_url. The URL of the media resource.
string (Conditionally required)Required when type is lip_sync_tts_speaker_id. TTS voice ID. Set to auto for random voice selection, or use a numeric string to specify a voice ID. See the table below for supported values.
Voice table
string (Conditionally required)Required when type is lip_sync_tts_content. Text content to drive lip sync. Supports both Chinese and English, maximum 200 characters (UTF-8 encoded).object (Optional)Video generation parameters, such as watermark settings.
Properties watermark boolean (Optional)Whether to add a watermark. The watermark is located at the bottom-right corner of the video with the fixed text "AI-generated".
|
Pass in a video and an audio file to generate a lip-synced video. |
Step 2: Query the result by task ID
Beijing region: GET https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}
- Polling recommendation: Video generation takes several minutes. We recommend using a polling mechanism with a reasonable query interval (such as 15 seconds) to obtain the result.
- Task status flow: PENDING (queued) → RUNNING (processing) → SUCCEEDED (success) / FAILED (failure).
- task_id validity: 24 hours. After expiration, the result cannot be queried and the API will return task status
UNKNOWN. - RPS limit: The default RPS for the query API is 20. For higher-frequency queries or event notifications, consider Configure asynchronous task callbacks.
- More operations: For batch queries, task cancellation, and other operations, see Manage asynchronous tasks.
Request parametersHeadersAuthorizationstring (Required)Authenticates the request with a Model Studio API key. Example: Bearer sk-xxxx.Path parameterstask_idstring (Required)The ID of the task. |
Replace {task_id} with the task_id value returned by the previous API call. The task_id is valid for queries for 24 hours, Replace {WorkspaceId} with your actual workspace ID. |
Response parameters |
|
outputobjectTask output information.
Properties task_id stringThe task ID. Valid for queries for 24 hours.task_status stringThe status of the task.
Enumeration values
stringThe time when the task was submitted. format is YYYY-MM-DD HH:mm:ss.SSS.scheduled_time stringThe time when the task was executed. format is YYYY-MM-DD HH:mm:ss.SSS.end_time stringThe time when the task was completed. format is YYYY-MM-DD HH:mm:ss.SSS.video_urlstringVideo URL. Returned only when task_status is SUCCEEDED.The video format is MP4 (H.264 encoding). The video link currently has no expiration time, but it is not recommended for long-term storage. Please download it promptly.orig_prompt stringThe original input prompt, corresponding to the request parameter prompt.code stringError code. Returned only for failed requests. See Error codes.message stringDetailed error message. Returned only for failed requests. See Error codes. | |
usage objectOutput statistics. Only counts successful results.
Properties duration integerBilling duration (seconds). For audio-driven mode: audio duration rounded up. For TTS mode: text byte count (UTF-8) / 15, rounded up.size stringOutput video resolution.fps integerOutput video frame rate.SR stringOutput video resolution tier.video_count integerNumber of videos. Fixed value: 1. |