Skip to main content

Integrate a Model Studio agent application

This topic describes how to integrate a Model Studio agent application into your multi-modal interaction application and call it from a client.

The multi-modal interaction development kit provides various official plugins and agents. If the official features do not meet your specific needs, you can integrate your own applications developed on the Model Studio platform, such as agent applications or workflows, to create richer multi-modal interactions. For recommended application templates, see Recommended Model Studio Application Templates. This document uses an example of creating and integrating an application named "Journey to the West Encyclopedia" to demonstrate how to integrate a Model Studio application into a multi-modal interaction application.

Create and configure a Model Studio application

  1. Create a Model Studio application. For more information, see Introduction to Application Types.
  2. Configure the application. Adjust the model, prompt, and any other capabilities you need. Rename the application to "Journey to the West Encyclopedia".
  3. Test and publish the application.
Select the Qwen-Turbo-Latest 1M model, and make sure to disable thinking mode. In the prompt area, write your instructions. You can use variables like ${user_name} to pass values from the client. In the skill configuration section, you can enable capabilities such as the MCP service, plugins, agents, and workflows as needed. After configuration is complete, test the application's behavior in the conversation area on the right.

Integrate into a multi-modal application

Import the "Journey to the West Encyclopedia" application you configured in the previous step into your multi-modal interaction application.
  1. In the menu, choose Agent > Add > My Applications. Select "Journey to the West Encyclopedia" and click OK to import it.
  2. Publish the application.
After you complete these steps, the imported "Journey to the West Encyclopedia" application appears in the Agent List of your multi-modal interaction application. Verify that the application status is Published.

Test the application and configure parameters

Test the application

After importing "Journey to the West Encyclopedia" into your multi-modal interaction application, you can test it on the web page. To activate the "Journey to the West Encyclopedia" application, use the voice command: "Open Journey to the West Encyclopedia and tell me who the Three Pure Ones are." The system successfully calls the Journey to the West Encyclopedia agent and responds with a child-friendly explanation of the Three Pure Ones in Taoism (Yuanshi Tianzun, Lingbao Tianzun, and Daode Tianzun). This verifies that the voice interaction test is successful.

Configure parameters

In the "Journey to the West Encyclopedia" application, we configured a parameter named ${user_name}. This represents the user's nickname and has a default value of "Xiaobao". When calling the multi-modal interaction application using the SDK, you can configure the value for ${user_name} in your code.

Level-1 parameter

Level-2 parameter

Level-3 parameter

Level-4 parameter

Description

biz_params

Business parameters for the multi-modal request.

user_defined_params

For passing user-defined parameters.

user_defined_app_id

The application ID of the imported Model Studio application.

user_prompt_params

Type: Object

The custom variable names and values in the prompt of the Model Studio application.

  • Format example
{
    "biz_params": {
        "user_defined_params": {
            "84***********************acc": {
                "user_prompt_params": {
                    "user_name": "Dami"
                }
            }
        }
    }
}

Update parameters using UpdateInfo

You can use this method to pass parameters to your agent during a conversation. These parameters are then included in the next request to the agent service. These parameters are included in every subsequent request to the agent. You can use UpdateInfo to update parameters with the same name.

parameters.bizParams

Parameter

Level-1 parameter

Type

Description

user_defined_params

Object

Parameters configured for the agent.

agentId

Object

An object where each key is an agent ID. The parameters in this object are passed to the agent.

user_prompt_params

Object

Use this object to pass parameters defined in the agent's prompt.

parameters.bizParams.user_defined_params 

"user_defined_params":{
    "xxxxx":{
        "user_prompt_params":{
            "age": 8,
            "text": "Today is January 9, 2025"
        }
    }
}
Example of setting parameters with the Android SDK 
private void updateParams(){
        try {
            JSONObject userAgentPromptParams = new JSONObject();
            userAgentPromptParams.put("age", 8);
            userAgentPromptParams.put("text","Today is January 9, 2025");
            JSONObject agentIdParams = new JSONObject();
            agentIdParams.put("user_prompt_params", userAgentPromptParams);
            JSONObject userDefinedParams = new JSONObject();
            userDefinedParams.put("tpa_2009506107285893120", agentIdParams); // The key is the agent ID.
            MultiModalRequestParam updateParams = MultiModalRequestParam
                    .builder()
                    .bizParams(MultiModalRequestParam.BizParams.builder()
                            .userDefinedParams(userDefinedParams)
                            .build())
                    .build();
            Log.i(TAG, "bizParams: " + updateParams.getParametersAsJson());
            multiModalDialog.updateInfo(updateParams.getParametersAsJson());
        } catch (JSONException e) {
            throw new RuntimeException(e);
        }
    }

Passing images using UpdateInfo

If your agent supports image modality interactions, you can also pass images to the agent as a URL or as base64-encoded data. Key parameter: parameters.images[]

images[]

Parameter

Type

Description

type

String

The type of the image. Valid values: "url" and "base64".

value

String

The image URL or base64-encoded data. Note: If you submit data in base64 format, the original image size cannot exceed 180 KB.

{
  "images": [{
    "type": "base64",
    "value": "\/9j\/4QDKRXhpZgAATU0AKgAAAAgABgESAAMAAAABAAEAAAEaAAUAAAABAAAAVgEbAAUAAAABAAAAXgEoAAMAAAABAAIAAAITAAMAAAABAAEAAIdpAAQAAAABAAAAZgAAAAAAAABIAAAAAQAAAEgBbEWenR5HJ\/lXXa62IW244FaR0RhN3ZwVp4fkljIRuleZeOtFu4oWAPavdtElAgbPYV5V46uh09KiT90qC1R80y2d---"
  }]
}
Example of setting parameters with the Android SDK 
MultiModalRequestParam updateParams = null;
try {
    updateParams = MultiModalRequestParam
    .builder()
    .images(MultimodalDialogActivity.this.getImageList())
    .build();
} catch (JSONException e) {
    e.printStackTrace();
}
multimodalDialog.updateInfo(updateParams);

Client-side verification

The following example uses the Android SDK.
  • Set the variable value in the connection establishment parameters.
HashMap<String, Object> appParams = new HashMap<>();
appParams.put("user_name","Dami");
HashMap<String, Object> userPromptParams = new HashMap<>();
userPromptParams.put("user_prompt_params",appParams);
HashMap<String, Object> userDefinedParams = new HashMap<>();
userDefinedParams.put("67f3ad7d6496475483db4a184c926e77",userPromptParams); // The application ID of the "Journey to the West Encyclopedia" application.
MultiModalRequestParam.BizParams bizParams = MultiModalRequestParam.BizParams
   .builder()
   .userDefinedParams(userDefinedParams)
   .build();
  • Run the demo and make a voice request to the "Journey to the West Encyclopedia" agent. Because the client set the username to "Dami", the application's reply addresses the user as "Dami". This verifies that the end-to-end test is successful.

Proactive exit from conversation takeover

In your multi-modal interaction application, you can enable "conversation takeover" mode for an integrated Model Studio application. When this mode is enabled and a user activates the application with a "trigger command", the application handles all subsequent turns in the conversation. This continues until the user says an "exit command" or the application itself proactively ends the conversation. In the Configure Application dialog box, enable the This application takes over all subsequent conversations switch. Set a trigger command (for example, "Open Journey to the West Encyclopedia") and an exit command (for example, "Exit Journey to the West Encyclopedia"), and then click OK. In addition to the user saying an exit command, a Model Studio application can proactively decide when to exit takeover mode based on its own business logic. This method provides more flexibility and allows for more intelligent conversation flow control. The application initiates a proactive exit by adding a specific "directive marker" to the beginning of its response. The following two exit scenarios are supported:
  • Scenario 1: The application exits after completing a task and providing a response This occurs when the application determines that the current conversation can end and has generated the final response for the turn. Once the application exits, the multi-modal platform plays only the returned response.
    • Directive marker: [#blmm-quit#] Example: The application returns [#blmm-quit#]Goodbye!. The user only hears Goodbye!. The conversation then exits the current application.
  • Scenario 2: The application exits after being unable to handle the current query and relinquishes control This occurs when the application determines it cannot handle the user's current request and must exit, returning the query to the main flow of the multi-modal interaction.
    • Directive marker: [#blmm-rejected#] Example: The application returns only [#blmm-rejected#]. The multi-modal platform takes over and finds another way to respond to the user's original query without playing any content from the current application.

Customize the broadcast voice

You can also specify a unique voice for your integrated Model Studio application to distinguish it from the voice of the main flow. In the Configure Application dialog box, disable the The application's broadcast voice remains consistent with the main flow switch. Then, in the voice drop-down list, select the desired voice (for example, Longan Huan).
  • In addition to specifying a voice for each integrated Model Studio application in the console, you can use API parameters for more granular voice control.

Parameter

Method

Required

Description

speaker_1_voice

SDK

No

Specifies the broadcast voice. The available values for the voice depend on the list of voices supported by the Text-to-Speech (TTS) model configured for the application.

  • Example of a client call. In the payload.biz_params node of the Start message, use the following format to set the broadcast voice:
{
  "user_defined_params": {
    "Replace with your Model Studio application ID": {
      "speaker_1_voice": "Replace with a voice supported by the TTS model configured for your Model Studio application"
    }
  }
}
  • How to find the Model Studio application ID
On the My Applications page of the Model Studio console, find the target application card. The Application ID displayed on the card is the ID you need.
Agent Development
Agent Context
Agent Governance & Evolution
Store
Application Marketplace
Use Cases
Support