Connect to an image generation agent over HTTP

Features

The image generation agent provides text-to-image, image-to-image, and scribble-to-image capabilities via HTTP. Currently, only pass-through links are supported. This service is ideal for use cases such as generating wallpapers, creating scribble art, enhancing photos, and stylizing images. A pass-through link sends a request directly to the image generation agent and returns the resulting image, bypassing nodes such as automatic speech recognition (ASR), intent recognition, and text-to-speech (TTS).

Integration

Prerequisites

Activate Alibaba Cloud Model Studio (Bailian) and obtain an API key. The API key authenticates your access to Alibaba Cloud Model Studio (Bailian). For instructions, see Obtain an API Key.

Console configuration

In the multi-modal development kit, create a multimodal interactive application. Select the All-in-one Edition (not the Vision Edition), disable Voice Interaction, and ensure Intent Recognition and Text Model remain enabled. For Text Model, select Multimodal Interaction Proprietary Model - High-speed Edition.
Disable Dialogue Context, Knowledge Base, Internet Search, and Long-term Memory. Set Number of Context Rounds to 5.
Clear all settings for Skills and MCP Service. From the list of agents, keep only the Image Generation Agent.
Configure the Image Generation Agent, which supports three features: Scribble to Image, Image Assistant, and text-to-image.
1. Scribble to Image: Generates images from hand-drawn scribbles or line art.
2. Image Assistant: Generates new images from an uploaded image.
3. Text-to-image: Generates images from text prompts.
Each feature supports options such as Model Selection, Prompts, Intelligent Optimization of Positive Prompts, and Negative Prompts. You can add variables to prompts to dynamically pass different content. For Scribble to Image, the Model Selection option provides three choices: Image-to-Image Advanced Edition, Image-to-Image Balanced Edition, and Image-to-Image Lightweight Edition.
After configuring the application, click the Publish button in the upper-right corner. You cannot test the application until it is published.

HTTP access

Request parameters

Parameter	Sub-parameter	Type	Required	Description
model		string	Yes	The name of the Model Studio model. Set this value to `"multimodal-dialog"`.
input	directive	string	Yes	The directive name. Set this to Request.
	app_id	string	Yes	Your application ID. You can find it on the "My Applications" page of the Multimodal Interaction Development Kit console. For more information, see Get an Application ID.
	dialog_id	string	No	The dialog ID. If you omit this parameter, the server starts a new dialog and returns a dialog ID. The ID is a 36-character string, such as `"12345678-1234-1234-1234-1234567890ab"`. To continue a previous conversation, pass the `dialog_id` returned by the server for that conversation. For image generation features that do not use conversational context, you can leave this parameter empty.
	text	string	Yes	The text to process. For this use case, if there is no text, set this parameter to an empty string (`""`).
parameters	client_info	object	Yes	For parameter details, see the Parameters: `parameters.client_info` table below.
	images	list[]	No	The reference images. Required for image-to-image scenarios, but optional for text-to-image scenarios. For parameter details, see the Parameters: `parameters.images` table below.
	biz_params	object	No	Configure this parameter as needed. For parameter details, see the Parameters: `parameters.biz_params` table below.

Parameters: `parameters.client_info`

Parameter	Sub-parameter	Type	Required	Description
user_id		string	Yes	The end user ID. Generate this ID based on your business rules. It enables customized features for different end users. Maximum length: 36 characters.
device	uuid	string	No	A globally unique ID for the client. Generate this ID and pass it to the SDK. Maximum length: 40 characters. An end user can have multiple devices, each with a different UUID but the same `user_id`.

Parameters: `parameters.images`

Parameter	Type	Required	Description
type	string	Yes	The type of the image. Valid values: `base64` and `url`.
value	string	Yes	The image content. If `type` is `base64`, this is the base64-encoded string of the image. If `type` is `url`, this is the URL of the image.

Parameter

Type

Required

Description

type

string

Yes

The type of the image. Valid values: base64 and url.

value

string

Yes

The image content.

If type is base64, this is the base64-encoded string of the image.
If type is url, this is the URL of the image.

Parameters: `parameters.biz_params`

Parameter	Sub-parameter	Type	Required	Description
commands[i]	name	string	Yes	Indicates a direct pass-through to the agent. Set this value to `"agent_command"`.
commands[i]	exec_params	object	Yes	Specifies the agent. This value is fixed for this use case. See the Parameters: `commands.exec_params` section.
user_defined_params	image_to_image	object	No	User-defined agent parameters. The `image_to_image` object contains parameters that are passed to the image generation agent. For this use case, populate this field only when passing custom variables for prompts to the agent.

Parameters: `parameters.biz_params.commands.exec_params`

Parameter	Sub-parameter	Type	Required	Description
exec_params	app_id	string	Yes	The agent to use. For this use case, set this to `"image_to_image"`.
	intent	string	Yes	Specifies the action for the agent. For this use case, set the value to `"open_image_to_image"`.
	slots	list[]	Yes	The slot information required by the agent.

Parameters: `parameters.biz_params.commands.exec_params.slots`

name	string	Yes	The name of the slot.
norm_value	string	Yes	The value of the slot.

For the image generation agent, you must populate the function and size slots as shown in the following example.

{
      "slots": [
          {
            "name": "function",
            "norm_value" :"imageAssistant"
          },
          {
            "name":"size",
            "norm_value":"768*768"
          }
      ]
}

The function parameter specifies the image generation feature to use. Supported values:

textToImage: text-to-image
imageAssistant: Image Assistant
sketchToImage: Sketch-to-image

The size parameter specifies the resolution of the generated image. The supported resolution range depends on the selected feature and model. See the following table for supported resolutions.

Image generation mode	Model configuration	Supported resolutions
text-to-image	Lightweight Edition	The image width and height must be between 200 and 2048 pixels. The aspect ratio must be between 1:4 and 4:1. For example, a resolution of 256×2048 is not allowed because its aspect ratio is 1:8.
	Balanced Edition
	Advanced Edition	Total pixels (width × height): Must be between 1,638,400 (1280×1280) and 2,073,600 (1440×1440). Image aspect ratio: Must be between 1:4 and 4:1.
Image Assistant/Sketch-to-image	Lightweight Edition	Image resolution: The width and height must both be between 512 and 4096 pixels.
	Balanced Edition	Total pixels (width × height): Must be between 262,144 (512×512) and 4,194,304 (2048×2048).
	Advanced Edition	Total pixels (width × height): Must be between 589,824 (768×768) and 4,194,304 (2048×2048).

Parameters: `parameters.biz_params.user_defined_params.image_to_image.user_prompt_params`

Populate this parameter only when passing custom variables for prompts to the image generation agent.

In the prompt area of the Sketch-to-image configuration panel, click the {x} Custom Variable button in the upper-right corner to add custom variables. ${style} is a default, built-in variable.

Parameter	Sub-parameter	Type	Required	Description
user_prompt_params	positive	object	No	Custom variables for the prompt.
user_prompt_params	negative	object	Yes	Custom variables for the negative prompt.

"user_defined_params": {
    "image_to_image": {
        "user_prompt_params": {
            "positive": {
                "hair": "long hair",
                "name": "Zhang San"
            },
            "negative": {
                "style": "low resolution"
            }
        }
    }
}

Text-to-image request example

{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
      "biz_params": {
        "commands": [
          {
            "name": "agent_command",
            "exec_params": {
              "app_id": "image_to_image",
              "intent": "open_image_to_image",
		"slots": [
		  {
		    "name": "function",
		    "norm_value" :"textToImage"
			},
		  {
		    "name":"size",
		    "norm_value":"768*768"
		        }
		]
            }
          }
        ],
	  "user_defined_params": {
		"image_to_image": {
                  "user_prompt_params":{
                    "positive":{
                       "hair": "long hair",
                       "name": "Zhang San"
                      },
                    "negative":{
                       "style": "low resolution"
                      }
                    }
		}
	  }
      }
    }
}

Text-to-image cURL request example

curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer your_api_key' \
  --header 'X-DashScope-SSE: enable' \
  --data '{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "textToImage"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "long hair",
                            "name": "Zhang San"
                        },
                        "negative": {
                            "style": "low resolution"
                        }
                    }
                }
            }
        }
    }
}'

Replace the following placeholders in the request: your_api_key (your API key) and app_id (your application ID).
Set the text field to an empty string (""). Otherwise, this triggers intent recognition.
Use the user_defined_params parameter to pass custom variables to the image generation agent's prompt.

Image assistant request

{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "images": [
            {
                "type": "url",
                "value": "img_url"
            }
        ],
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "imageAssistant"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "long hair",
                            "name": "Zhang San"
                        },
                        "negative": {
                            "style": "low resolution"
                        }
                    }
                }
            }
        }
    }
}

Image assistant cURL request

curl --location "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation" \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer your_api_key' \
  --header 'X-DashScope-SSE: enable' \
  --data '{
    "model": "multimodal-dialog",
    "input": {
        "directive": "Request",
        "app_id": "xxxxxx",
        "text": ""
    },
    "parameters": {
        "images": [
            {
                "type": "url",
                "value": "img_url"
            }
        ],
        "client_info": {
            "user_id": "test-251222",
            "device": {
                "uuid": "test-251222-123"
            }
        },
        "biz_params": {
            "commands": [
                {
                    "name": "agent_command",
                    "exec_params": {
                        "app_id": "image_to_image",
                        "intent": "open_image_to_image",
                        "slots": [
                            {
                                "name": "function",
                                "norm_value": "imageAssistant"
                            },
                            {
                                "name": "size",
                                "norm_value": "768*768"
                            }
                        ]
                    }
                }
            ],
            "user_defined_params": {
                "image_to_image": {
                    "user_prompt_params": {
                        "positive": {
                            "hair": "long hair",
                            "name": "Zhang San"
                        },
                        "negative": {
                            "style": "low resolution"
                        }
                    }
                }
            }
        }
    }
}'

Replace your_api_key with your API key, app_id with your app ID, and img_url with the image URL.
Set the text field to an empty string (""). Otherwise, the request is routed for intent recognition.
Use the user_defined_params parameter only when passing custom variables to the agent prompt for image generation.

Sketch-to-image request

Use the Image Assistant request example, changing the value of the function slot in parameters.biz_params.commands.exec_params.slots from imageAssistant to sketchToImage.

The `RespondingContent` event

Top-level parameter	Second-level parameter	Type	Required	Description
output	event	string	Yes	Event name: RespondingContent
	dialog_id	string	Yes	A unique identifier for the dialog.
	round_id	string	Yes	A unique identifier for the current round of interaction.
	llm_request_id	string	Yes	The request ID for the call to the large language model (LLM).
	text	string	Yes	The URL of the generated image. The value is `RUNNING` while generation is in progress and becomes the image URL upon completion.
	spoken	string	Yes	Content for Text-to-Speech (TTS) synthesis. Ignore this field if TTS is not enabled. In the examples in this guide, its value is the same as the `text` field.
	finished	bool	Yes	Indicates whether the output stream has finished.
	finish_reason	string	No	The reason the output stream finished. Currently, only one value is supported: `stop`: Indicates that the stream completed normally.
	extra_info	object	No	Additional information. Currently, the following field is supported: `agent_info`: Information about the agent.

Parameters for `output.extra_info.agent_info`

Top-level parameter	Second-level parameter	Type	Required	Description
round		string	Yes	The dialog round number.
device	device_id	string	Yes	Corresponds to the `device.uuid` from the request.
intent_infos	intent	string	Yes	The intent being executed. For the examples in this guide, the value is always "open_image_to_image".
	domain	string	Yes	The domain of the intent. For the examples in this guide, the value is always "image_to_image".

Response example

{
    "output": {
        "round_id": "xxx",
        "llm_request_id": "xxx",
        "extra_info": {
            "agent_info": {
                "round": 1,
                "device": {
                    "device_id": "test-251222-123"
                },
                "intent_infos": [
                    {
                        "intent": "open_image_to_image",
                        "domain": "image_to_image"
                    }
                ]
            },
            "query": ""
        },
        "dialog_id": "6c4340eb-xxx-4055-8d66-0794df7986e0",
        "spoken": "img_url",
        "finished": true,
        "text": "img_url",
        "event": "RespondingContent"
    },
    "request_id": "xxx"
}

During image generation, the service sends a heartbeat every 3–5 seconds. While in progress, output.text is RUNNING and output.finished is false.
When image generation completes, output.finished becomes true, and output.text contains the image URL.

id:1 Generation initiated
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84a8xxx","llm_request_id":"f22feed889994756b3787de8b08bxxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}]},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5ca9xxx","spoken":"","finished":false,"text":"","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:2 Generation in progress
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8b0xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5ca9xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb7xxx"}
id:3 Generation in progress
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfe84xxx","llm_request_id":"f22feed889994756b3787de8bxxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-7c5cxxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcb71xxx"}
id:4 Generation in progress
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbb1xxx","llm_request_id":"f22feed889994756b3787xxx","extra_info":{"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"dialog_id":"00cd961c-bf58-49f5-b3bb-xxx","spoken":"RUNNING","finished":false,"text":"RUNNING","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fclxxx"}
id:5 Generation complete
event:result
:HTTP_STATUS/200
data:{"output":{"round_id":"6633eccfd6a44090b49bbbfxxx","llm_request_id":"f22feed889994756b3787de8xxx","extra_info":{"tool_infos":[{"output":"","tool_name":"image_to_image","arguments":{},"type":"agent","success":true}],"agent_info":{"round":1,"device":{"device_id":"xxx-test-251222-123"},"intent_infos":[{"intent":"open_image_to_image","domain":"image_to_image"}]},"query":""},"finish_reason":"stop","dialog_id":"00cd961c-bf58-49f5-b3bb-7cxxx","spoken":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","finished":true,"text":"https://dashscope-result-sh.oss-cn-shanghai.aliyuncs.com/1d/03/20260107/c6e3cb4b/eb8e554d-66ff-44ca-ba0d-f05481405d2c-1-1.png?Expires=1767854752&OSSAccessKeyId=YOUR_ACCESS_KEY_ID&Signature=YOUR_SIGNATURE","event":"RespondingContent"},"request_id":"a162bf16-8156-4b13-88f6-fcbxxx"}

​Features

​Integration

​Prerequisites

​Console configuration

​HTTP access

​Request parameters

​Parameters: parameters.client_info

​Parameters: parameters.images

​Parameters: parameters.biz_params

Parameters: parameters.biz_params.commands.exec_params

Parameters: parameters.biz_params.commands.exec_params.slots

Parameters: parameters.biz_params.user_defined_params.image_to_image.user_prompt_params

​Text-to-image request example

​Text-to-image cURL request example

​Image assistant request

​Image assistant cURL request

​Sketch-to-image request

​The RespondingContent event

​Parameters for output.extra_info.agent_info

​Response example

Features

Integration

Prerequisites

Console configuration

HTTP access

Request parameters

Parameters: `parameters.client_info`

Parameters: `parameters.images`

Parameters: `parameters.biz_params`

Parameters: `parameters.biz_params.commands.exec_params`

Parameters: `parameters.biz_params.commands.exec_params.slots`

Parameters: `parameters.biz_params.user_defined_params.image_to_image.user_prompt_params`

Text-to-image request example

Text-to-image cURL request example

Image assistant request

Image assistant cURL request

Sketch-to-image request

The `RespondingContent` event

Parameters for `output.extra_info.agent_info`

Response example