This topic describes how to use an API to call a legacy Alibaba Cloud Model Studio retrieval-augmented generation (RAG) application.
Migration plan
The enterprise knowledge base feature is no longer maintained. We recommend that you migrate your private knowledge to the new knowledge base as soon as possible. Alibaba Cloud does not currently provide official migration tools or services. The new knowledge base offers more features and supports more document formats.
| Enterprise knowledge type | Migration plan |
|---|---|
| Documents | Create an unstructured knowledge base to manage document-based knowledge. Import documents by uploading them from your local machine or from Object Storage Service (OSS).If you upload from a local machine, you must manually update the knowledge base later. If you import from OSS, you can integrate OSS, Function Compute (FC), and the Alibaba Cloud Model Studio knowledge base API to enable automatic data updates.You can upload local files using the console or an API. You can only import from OSS using the console.For more information, see Knowledge base: Step 1. Import data. |
| FAQ | Create a structured knowledge base to manage FAQ-style knowledge. You can create a structured knowledge base by uploading local documents or using ApsaraDB RDS.If you upload local documents, you must manually update the knowledge base later. If you use ApsaraDB RDS, data updates in the RDS table are automatically synchronized to the knowledge base.Currently, both methods can only be performed in the console.For more information, see Knowledge base: Step 1. Import data. |
Use the SDK
Prerequisites
- You have obtained an API key and configured it as an environment variable. For more information, see Obtain an API key and Configure an API key.
- The DashScope SDK is installed. Python and Java versions are available. For more information, see Install the latest version of the SDK.
- A RAG application is created and enterprise knowledge is uploaded. You can no longer create new legacy RAG applications. For more information, see Upload enterprise knowledge.
- Obtain the APP_ID. You can view it on the application card on the My Applications page.
Call examples
Retrieve by tag
If your enterprise knowledge base contains a large volume of data, use tag-based retrieval to improve recall accuracy.
First, add knowledge tags to your documents. For more information, see Upload enterprise knowledge.
Then, obtain the tag IDs and pass them in the doc_tag_codes parameter. You can pass multiple tag IDs.
Multi-turn conversation
Cloud-hosted multi-turn conversation
Alibaba Cloud Model Studio applications provide a cloud-hosted multi-turn conversation feature. Use the session_id to conduct multi-turn conversations. Alibaba Cloud automatically manages the conversation history, so you do not need to maintain it on the client side.
In the following example, the first call returns a session_id. When you pass this session_id in the second call, the model service includes the session information from the first call.
session_id and history, the history parameter takes precedence, and the cloud-hosted conversation is not used.Client-side multi-turn conversation
If you maintain the conversation history on the client side, you can pass the conversation information in the history parameter.
Streaming output
To enable streaming output, add the corresponding parameter. In the Python SDK, add stream=True. In the Java SDK, use the streamCall interface.
Workspaces
The preceding examples call an application in the default workspace. To call an application in a different workspace, you must pass the specific workspace ID.
YOUR_WORKSPACE in the example with your actual workspace ID to ensure the code runs correctly. To obtain the workspace ID, see Obtain a workspace ID.Input parameters
Parameter | Type | Default | Description |
|---|---|---|---|
app_id | string | - | The application ID. |
prompt | string | - | The instruction that you want the model to execute. This guides the model in generating a response. |
session_id | string | - | The unique ID of a conversation session. If you pass a session_id, the conversation history is recorded in the cloud. The LLM automatically includes the stored conversation history in subsequent calls. Make sure that the session_id is unique. You can use either session_id or history, but not both. |
history (optional) | list[dict] | [] | The conversation history between the user and the model. Each element in the list is a turn in the conversation, formatted as {"user": "user input", "bot": "model output"}. The turns are arranged chronologically. |
workspace (optional) | string | - | The workspace ID. |
seed (optional) | int | 1234 | The random number seed used for generation. This controls the randomness of the model's output. The seed must be an unsigned 64-bit integer. The default value is 1234. When a seed is used, the model tries to produce the same or similar results, but identical results are not guaranteed for every generation. |
top_p (optional) | float | 0.2 | The probability threshold for nucleus sampling. For example, a value of 0.2 means that only the smallest set of most-likely tokens with a cumulative probability of 0.2 or higher are kept as the candidate set. The value must be in the range of (0, 1.0). A higher value increases randomness, while a lower value increases determinism. |
top_k (optional) | int | None | The size of the candidate set for sampling. For example, a value of 50 means that only the 50 tokens with the highest scores are included in the candidate set for random sampling. A higher value increases randomness, while a lower value increases determinism. If this parameter is not passed, is set to None, or is greater than 100, the top_k policy is not used, and only the top_p policy is in effect. |
temperature (optional) | float | 1.0 | Controls the degree of randomness and diversity. The temperature value smooths the probability distribution of candidate words. A higher temperature value flattens the distribution, allowing less probable words to be selected and making the output more diverse. A lower temperature value sharpens the distribution, making more probable words more likely to be selected and making the output more deterministic. Valid values: [0, 2). A value of 0 is not recommended because it is meaningless. python version >=1.10.1 java version >= 2.5.1 |
stream (optional) | bool | False | Specifies whether to use streaming output. When streaming is enabled, the API operation returns a generator. You must iterate through the generator to get the results. By default, each output is the entire sequence generated so far, and the final output is the complete result. You can set the incremental_output parameter to False to change to non-incremental output mode. |
doc_tag_codes (optional) | list[str] | - | A list of document tag codes. When you pass a list of tag codes, the retrieval process is limited to the documents associated with those tags. |
doc_reference_type | str | - | The reference data type for the retrieved documents. Set the value to `simple`. If you set the value to `simple`, the response does not contain reference markers. |
has_thoughts | bool | False | Specifies whether to output information about the retrieval and recall process. If enabled, the response includes details about the document retrieval, recall, and model inference processes. |
Output parameters
Response parameter | Type | Description | Remarks |
|---|---|---|---|
status_code | int | A value of 200 (HTTPStatus.OK) indicates that the request was successful. Otherwise, the request failed. You can get the error code from the `code` field and the detailed error message from the `message` field. Note This field is for Python only. In Java, a failure throws an exception that contains the code and message. | |
request_Id | string | The ID generated by the system for this call. | |
code | string | The error code if the request fails. This is ignored on success. Python only | |
message | string | The detailed error message if the request fails. This is ignored on success. Python only | |
output | dict | The call result. For Qwen models, this includes the output text. | |
usage | dict | The metering data for this request. | |
output.text | string | The response generated by the model. | |
output.finish_reason | string | This is null during generation. If generation stops due to a stop token, the value is `stop`. | |
output.session_id | string | The unique ID of the conversation session. | In a multi-turn conversation, you can use this ID for session persistence. |
usage.models[].model_id | string | The model invoked by the application for this call. | |
usage.models[].input_tokens | int | The number of tokens in the user input text. | |
usage.models[].output_tokens | int | The number of tokens in the response generated by the model. | |
output.thoughts[].throught | string | The model's thought process result. | |
output.thoughts[].action_type | string | The type of execution step returned by the LLM. `api`: Execute an API plugin. `response`: Return the final result. | |
output.thoughts[].action_name | string | The name of the action being executed, such as document retrieval or an API plugin. | |
output.thoughts[].action | string | The step being executed. | |
output.thoughts[].action_input_stream | string | The streaming result of the input parameter. | |
output.thoughts[].action_input | string | The input parameters for the plugin. | |
output.thoughts[].response | string | The result returned by the model call. | |
output.thoughts[].observation | string | The result returned by the retrieval or plugin. |
HTTP API
Description
The application supports API calls that use the standard HTTP and HTTP Server-Sent Events (SSE) protocols. You can choose the protocol that suits your needs.
Prerequisites
- The Alibaba Cloud Model Studio service is activated. For more information, see Activate the service.
- An API key is created. For more information, see Obtain an API key.
- A RAG application is created and enterprise knowledge is uploaded. For more information, see Build a Q&A application with no code and Upload enterprise knowledge.
Submit an API call
Request parameters
Parameter-passing method | Field | Type | Required | Description | Example |
|---|---|---|---|---|---|
Header | Content-Type | String | Yes | The request type: application/json | application/json |
Accept | String | No | */*. Set to `text/event-stream` to enable SSE responses. The default is not set. | text/event-stream | |
Authorization | String | Yes | The API key. Example: Bearer d1**2a | Bearer d1**2a | |
X-DashScope-SSE | String | No | Use either this or `Accept: text/event-stream` to enable SSE responses. | enable | |
X-DashScope-WorkSpace | String | No | The workspace ID. | ws_ik******RVYCKzt | |
Body | input.prompt | String | Yes | The instruction that you want the model to execute. Both Chinese and English are supported. | Which park is closer to me? |
input.history | List | No | The conversation history between the user and the model. Each element in the list is a turn in the conversation, formatted as {"user": "user input", "bot": "model output"}. The turns are arranged chronologically. | "history": [ { "user":"How is the weather today?", "bot":"The weather is nice today. Do you want to go out?" }, { "user":"Do you have any recommendations?", "bot":"I suggest going to the park. Spring is here, and the flowers are beautiful." } ] | |
input.session_id | String | No | The unique ID of a conversation session. If you pass a session_id, the conversation history is recorded in the cloud. The LLM automatically includes the stored conversation history in subsequent calls. Make sure that the session_id is unique. You can use either session_id or history, but not both. | ||
input.doc_tag_codes | String | No | A list of document tag codes. When you pass a list of tag codes, the retrieval process is limited to the documents associated with those tags. | ["471d*******3427", "881f*****0c232"] | |
parameters.seed | Integer | No | The random number seed used for generation. This controls the randomness of the model's output. The seed must be an unsigned 64-bit integer. The default value is 1234. When a seed is used, the model tries to produce the same or similar results, but identical results are not guaranteed for every generation. | 65535 | |
parameters.top_p | Float | No | The probability threshold for nucleus sampling. For example, a value of 0.2 means that only the tokens with a cumulative probability of 0.2 or higher are kept as the candidate set for random sampling. The value must be in the range of (0, 1.0). A higher value increases randomness, while a lower value decreases randomness. The default value is 0.2. Do not set the value to 1 or higher. | 0.2 | |
parameters.top_k | Integer | No | The size of the candidate set for sampling. For example, a value of 50 means that only the 50 tokens with the highest scores are included in the candidate set for random sampling. A higher value increases randomness, while a lower value increases determinism. Note: If the top_k parameter is empty or its value is greater than 100, the top_k policy is not used, and only the top_p policy is in effect. The default is empty. | 50 | |
parameters.temperature | Float | No | Controls the degree of randomness and diversity. The temperature value smooths the probability distribution of candidate words. A higher temperature value flattens the distribution, allowing less probable words to be selected and making the output more diverse. A lower temperature value sharpens the distribution, making more probable words more likely to be selected and making the output more deterministic. Valid values: [0, 2). The system default is 1.0. A value of 0 is not recommended because it is meaningless. | 1.0 | |
parameters.has_thoughts | Bool | No | Specifies whether to output information about the retrieval and recall process. If enabled, the response includes details about the document retrieval, recall, and model inference processes. |
Response parameters
Field | Type | Description | Example |
|---|---|---|---|
status_code | int | A value of 200 (HTTPStatus.OK) indicates that the request was successful. Otherwise, the request failed. You can get the error code from the `code` field and the detailed error message from the `message` field. Note This field is for Python only. In Java, a failure throws an exception that contains the code and message. | 200 |
request_Id | string | The ID generated by the system for this call. | 33dcf25a-******-8b711f15614e |
code | string | The error code if the request fails. This is ignored on success. | |
message | string | The detailed error message if the request fails. This is ignored on success. | |
output.text | string | The response generated by the model. | |
output.finish_reason | string | This is null during generation. If generation stops due to a stop token, the value is `stop`. | |
output.session_id | string | The unique ID of the conversation session. | In a multi-turn conversation, you can use this ID for session persistence. |
usage.models[].model_id | string | The model invoked by the application for this call. | |
usage.models[].input_tokens | int | The number of tokens in the user input text. | |
usage.models[].output_tokens | int | The number of tokens in the response generated by the model. | |
output.thoughts[].throught | string | The model's thought process result. | |
output.thoughts[].action_type | string | The type of execution step returned by the LLM. `api`: Execute an API plugin. `response`: Return the final result. | |
output.thoughts[].action_name | string | The name of the action being executed, such as document retrieval or an API plugin. | |
output.thoughts[].action | string | The step being executed. | |
output.thoughts[].action_input_stream | string | The streaming result of the input parameter. | |
output.thoughts[].action_input | string | The input parameters for the plugin. | |
output.thoughts[].response | string | The result returned by the model call. | |
output.thoughts[].observation | string | The result returned by the retrieval or plugin. |
Request example (SSE disabled)
The following example shows a script that uses a cURL command to call a RAG application with SSE disabled.
YOUR_API_KEY with your API key and YOUR_APP_ID with your app ID to ensure the code runs correctly.Response example (SSE disabled)
Request example (SSE enabled)
The following example shows a script that uses a cURL command to call a RAG application with SSE enabled.
YOUR_API_KEY with your API key and YOUR_APP_ID with your app ID to ensure the code runs correctly.Response example (SSE enabled)
Request example (workspace)
The following example shows a script that uses a cURL command to call a RAG application in a specified workspace.
YOUR_API_KEY with your API key, YOUR_APP_ID with your app ID, and YOUR_WORKSPACE with your workspace ID to ensure the code runs correctly.Response example (workspace)
Error response example
If an error occurs during the request, the output includes a code and a message that indicate the cause of the error.