The key interfaces and request parameters for Qwen-Omni real-time using the DashScope Python SDK.
For connection and authentication, see WebSocket access. For event fields and JSON examples, see Client events and Server events.
Your SDK version must be 2.0.1 or later. Before you start, see Real-time multimodal interaction flow.
To use
The following request parameters can be set with the
Configure the following request parameters with the update_session method.
The real-time session is created with
Use the
The server returns response events and data in the form of events. You need to iterate the session's
Prerequisites
Your SDK version must be 2.0.1 or later. Before you start, see Real-time multimodal interaction flow.
Quick start
To use qwen3.8-omni-flash-realtime, set the session constructor's model parameter to this model name. The default voice is Tina. For Python SDK session configuration and video aggregation examples, see the real-time guide.
Download the sample code from GitHub. Three calling methods are available:
-
Audio conversation example: Captures real-time audio input from the microphone, enables VAD mode to automatically detect the start and end of speech, and supports voice interruption.
enable_turn_detectionparameter must be set to True.We recommend that you use headphones for audio playback to prevent echoes from triggering voice interruption.
-
Audio and video conversation example: Captures real-time audio and video input from the microphone and camera, enables VAD mode to automatically detect the start and end of speech, and supports voice interruption.
enable_turn_detectionparameter must be set to True.We recommend that you use headphones for audio playback to prevent echoes from triggering voice interruption.
-
Local call: Uses local audio and images as input and enables Manual mode, which lets you manually control the sending pace.
enable_turn_detectionparameter must be set to False.
Request parameters
The following request parameters can be set with the client.audio.realtime.connect method. Server-side events are no longer delivered via callbacks. Instead, iterate the session's events() in a receiver thread and dispatch by event["type"] (see Event iteration).
Parameter | Type | Description |
|---|---|---|
model | str | The Qwen-Omni model to use. See Model list. |
ws_url | str | The full WebSocket endpoint. If not specified, the endpoint is automatically selected based on the
Replace |
| Parameter | Type | Description |
|---|---|---|
| output_modalities | list[str] | The model output modality. Set to ['text'] for text only, or ['text', 'audio'] for both audio and text. |
| voice | str | The voice for audio output. For supported voices, see Voice list.Default voices:
|
| input_audio_config | dict | Input audio format configuration. Use a dict (with type and sample_rate keys) to set type (pcm or wav) and sample_rate (8000, 16000, 24000, or 48000; default: 16000 Hz) together. For usage, see Audio format and sample rate configuration.Applies to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime. |
| output_audio_config | dict | Output audio format configuration. Use a dict (with type and sample_rate keys) to set type (pcm or wav) and sample_rate (8000, 16000, 24000, or 48000) together. If no sample rate is specified, 16000 Hz is used by default; explicitly set the desired output rate, such as 24000 Hz.Applies to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime. |
| input_audio_format | str | Legacy compatibility field. For new integrations, we recommend that you use input_audio_config to set the format and sample rate together. |
| output_audio_format | str | Legacy compatibility field. For new integrations, we recommend that you use output_audio_config to set the format and sample rate together. |
| smooth_output | bool | This parameter is supported only by qwen3-omni-flash-realtime-2025-09-15.
|
| instructions | str | A system message that sets the model's objective or role.For example: You are an AI agent for a five-star hotel. Answer customer questions about room types, facilities, prices, and booking policies. Be accurate and friendly. Always respond with a professional and helpful attitude. Do not provide unverified information or information outside the hotel's scope of services. |
| enable_input_audio_transcription | bool | Enables speech recognition for input audio. |
| input_audio_transcription_model | str | Speech recognition model for input audio transcription. The value is always qwen3-asr-flash-realtime. This parameter is not configurable. |
| turn_detection_type | str | The Voice Activity Detection (VAD) type. Valid values:
|
| turn_detection_threshold | float | VAD detection threshold. Increase in noisy environments and decrease in quiet environments.
|
| turn_detection_silence_duration_ms | int | Silence duration that indicates end of speech. If exceeded, the model triggers a response. Default: 800. Valid values: [200, 6000]. |
| turn_detection_param | dict | Additional turn_detection configuration parameters. Currently supports idle_timeout_ms (int): the idle timeout in milliseconds. Applies only toqwen3.5-omni-plus-realtimeandqwen3.5-omni-flash-realtimemodels inserver_vadmode. After the server finishes audio playback and the user remains silent beyond this duration (no speech.started triggered), the model proactively generates a response to prompt the user to continue the conversation. Valid range: [5000, 30000].Example: turn_detection_param={'idle_timeout_ms': 5000} |
| enable_search | bool | Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models.Enables web search. Set to true to enable. Default: false. When enabled, the model can autonomously decide whether to search the web to answer users' real-time questions.Tool calling (tools) and web search (enable_search) are incompatible. You cannot enable both at the same time. |
| search_options | object | Web search options. Takes effect only when enable_search is enabled.Currently, you can only set enable_source (Boolean), which controls whether to return search result sources. Set to true to enable. Example: search_options={'enable_source': True}. |
| tools | list[dict] | Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models.Tool definitions. When provided, the model can call external tools to respond to user questions. For Function Calling, the model does not generate audio and only returns the tool calling parameters.Qwen3.8-Omni-Flash-Realtime supports Function Calling and MCP tools in the same session. See client events for MCP configuration and MCP call limits for quotas. tools and enable_search are mutually exclusive, including MCP tools.The following fields describe Function Calling tools. Each tool is a dictionary that contains the following fields:
|
| temperature | float | Sampling temperature that controls content diversity.Higher values produce more diverse content; lower values produce more deterministic content.Valid values: [0, 2).Because both temperature and top_p control content diversity, we recommend that you set only one of them.
|
| top_p | float | Probability threshold for nucleus sampling that controls content diversity.Higher values produce more diverse content; lower values produce more deterministic content.Valid values: (0, 1.0].Because both temperature and top_p control content diversity, we recommend that you set only one of them.Default top_p values:
|
| top_k | integer | Candidate set size for sampling. For example, 50 means only the 50 highest-scoring tokens form the candidate set. Larger values increase randomness; smaller values increase determinism. Set to None or a value greater than 100 to disable top_k and use only top_p.The value must be greater than or equal to 0.Default top_k values:
|
| max_tokens | integer | Maximum number of tokens to return.For qwen3.8-omni-flash-realtime, the valid range is [1, 65536]. For other models, default and maximum values equal the model's maximum output length. See the Model Studio console for details.Use max_tokens to limit output length for generating summaries, keywords, controlling costs, or reducing response time. |
| repetition_penalty | float | Controls repetition in generated sequences. Higher values reduce repetition. 1.0 means no penalty. qwen3.8-omni-flash-realtime also accepts 0. For other models, the value must be greater than 0.Default repetition_penalty values:
|
| presence_penalty | float | Controls content repetition in model output.Valid values: [-2.0, 2.0]. Positive values reduce repetition; negative values increase it.Default presence_penalty values:
|
| seed | integer | Makes generation more deterministic, ensuring consistent results across runs.Passing the same seed with identical parameters produces the same result as much as possible.Valid values: 0 to 231−1. Default value: -1. |
API reference
RealtimeSession (client.audio.realtime.connect)
The real-time session is created with client.audio.realtime.connect(model=...). The connection is established when the with context is entered (or when the session's connect() method is called).
| Method signature | Server-side response event (delivered via events()) | Description |
|---|---|---|
Server-side eventSession createdsession.updated Session configuration updated | Creates a connection with the server. This method is automatically called when the with context is entered. Do not call connect() first and then enter the with context of the same session. | |
session.updatedSession configuration updated | Updates session configuration. For parameter details, see Request parameters.After connecting, the server returns default session configurations. Call this method immediately after connecting to update settings.When the server receives the session.update event, it validates the parameters. Invalid parameters return an error; otherwise the server updates session configuration. | |
| None | Appends audio data to the cloud input buffer. str is treated as Base64-encoded data; bytes is automatically Base64-encoded. The buffer is temporary storage that can be written to and committed later.
| |
| None | Adds image data to the cloud video buffer. str is treated as Base64-encoded data; bytes is automatically Base64-encoded. Accepts local images or real-time video stream captures.Image input limits:
| |
input_audio_buffer.clearedDeletes the audio received by the server | Deletes the audio from the current cloud buffer. | |
input_audio_buffer.committedServer received the committed audio | Commits audio and video from the cloud buffer. Returns an error if the buffer is empty.
| |
Server-side eventServer starts generating a responseresponse.output_item.added New output content is available in the responseServer-side event Conversation item createdresponse.content_part.added New output content added to the assistant message itemresponse.audio_transcript.delta Incrementally generated transcribed textresponse.audio.delta Incrementally generated audio from the modelresponse.audio_transcript.done Text transcription completedresponse.audio.done Audio generation completedresponse.content_part.done Streaming of text or audio content for the assistant message is completeresponse.output_item.done Streaming of the entire output item for the assistant message is completeresponse.done Response completed | Instructs the server to create a model response.When you configure a session in "turn_detection" mode, the server automatically creates a model response. | |
| None | Cancels the in-progress response. If there is no response to cancel, the server responds with an error. | |
| None | Sends a conversation.item.create event to the server.For a Function Calling tool result, item is a dictionary with these fields:
item.type="mcp_approval_response". See MCP approval response for its fields. | |
| None | Terminates the task and closes the connection (idempotent; does not send session.finish). | |
| None | Gets the session_id of the current task and the response_id of the last response (session attributes; additional first-packet delay metrics such as last_first_text_delay/last_first_audio_delay are also available). |
Audio format and sample rate configuration
Use the input_audio_config and output_audio_config parameters of update_session (dicts with type and sample_rate keys) to set the audio format and sample rate together.
Applies to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime.
Event iteration (events())
The server returns response events and data in the form of events. You need to iterate the session's events() in a receiver thread and dispatch by event["type"] (see Server-side events for the complete event list).
| Method signature | Parameters | Description |
|---|---|---|
| event: Server-side event | Continuously reads and yields server-side events (dict) one by one. The iteration ends when the connection is closed or session.finished is received. Use a single receiver thread to consume events continuously; do not create multiple concurrent readers. | |
| None | Blocking read of a single server-side event, without automatic looping. Returns None when there is no event (connection closed). Error events are raised as DashScopeError exceptions. |