Skip to main content

Multi-turn Conversations: Correctly Passing Tool Call History

End-to-end practice—from knowledge base creation to multi-turn API calls—demonstrating how to pass tool call history to avoid redundant retrieval

This guide demonstrates how to pass tool call history across multi-turn conversations, enabling the model to answer continuously based on prior retrieval results and avoid redundant searches. All steps use the same sample document set for reproducible results.

Principle: Why Context Is Lost

RAG Agents are stateless services: each request is independent, and the server does not retain conversation context. A single response typically involves multiple stages: 
User Query ──→ Reasoning ──→ Tool Call for Retrieval ──→ Retrieval Results ──→ Sufficient Information?
                  ▲                                                                │
                  └────────────────────────── No ──────────────────────────────────┤
                                                                                   │ Yes
                                                                                   ▼
                                                                      Final Answer Generation
If only textual history (user query + assistant reply) is passed in the next turn, the model cannot see intermediate tool calls or know what has already been retrieved—leading to redundant retrieval of the same question. This wastes tokens and may yield inconsistent answers. The correct approach is to include the full tool call chain (assistant.tool_calls + corresponding tool return results) as part of the message history, allowing the model to build upon existing retrieval results.

Prepare Sample Documents

This guide uses the Bailian Technical Documentation Sample, containing 92 official technical documents from the Bailian platform (API usage, model descriptions, best practices, etc.). Download, extract, and upload them to your knowledge base to begin. For demonstration, we’ll use the document “First-Time Calling of Qwen API” to illustrate multi-turn follow-up:
  • Turn 1: “How do I call the Qwen API?” (triggers multi-step tool-based retrieval)
  • Turn 2: “How do I configure my API Key as an environment variable?” (verifies whether history is reused and redundant retrieval is avoided)
You may also use your own documents—the steps and outcomes remain similar.

End-to-End Workflow

1. Create a Knowledge Base and Upload Sample Documents

  1. Go to Data Ingestion → Knowledge Management and click Create Knowledge Base
  2. Select Document Search as the knowledge base type and Basic Document Q&A as the use case
  3. Upload the sample documents, complete chunking and vectorization, and wait until the document status changes to Parsing Completed
See Creating a Knowledge Base and Document Management for details.

2. Create a Knowledge Q&A Service and Bind the Knowledge Base

  1. Go to Knowledge Services → Knowledge Q&A and click Create Q&A Service
  2. Enter a service name and proceed to the configuration page after successful creation
  3. Click + Add, then bind the knowledge base created in Step 1
  4. Select a generation model and retrieval mode (Multi-turn Intelligent Retrieval is recommended for multi-turn conversations), then click Publish
See Knowledge Q&A for details.

3. Retrieve API Parameters

From the Q&A service list, click API Debugging for your target service to obtain the following parameters:
ParameterWhere to Find ItExample
endpointInterface URL on the API Debugging page; {workspace_id} is your workspace IDhttps://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat
agent_idRequest parameter on the API Debugging page—the Q&A service IDaid-xxxxxxxx
API KeyCreated on the Settings → API Key pagesk-xxxxxxxx
agent_id uniquely identifies a knowledge Q&A service—each service corresponds to one agent_id.

4. First Turn API Call

Use the retrieved parameters to make your first request. The request structure is as follows: 
curl -X POST 'https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat' \
  -H 'Authorization: Bearer $API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "input": {
      "messages": [{"role": "user", "content": "How do I call the Qwen API?"}]
    },
    "parameters": {
      "agent_options": {"agent_id": "aid-xxxxxxxx"}
    },
    "stream": true
  }'
Internally, the Agent executes multi-step tool calls to retrieve from the knowledge base and returns the final answer via streaming. You must extract the tool call history from the SSE stream—see Extracting Tool History from SSE Stream below.

5. Second Turn: Follow-Up with Full Tool History

For the second query—“How do I configure my API Key as an environment variable?”—pass the full history from Turn 1, including the tool call chain: 
curl -X POST 'https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat' \
  -H 'Authorization: Bearer $API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "input": {
      "messages": [
        {"role": "user", "content": "How do I call the Qwen API?"},
        {"role": "assistant", "content": "", "tool_calls": [{"id":"call_956...","function":{"name":"semantic_search",...}}]},
        {"role": "tool", "tool_call_id": "call_956...", "content": "Retrieved 5 relevant chunks..."},
        {"role": "assistant", "content": "Based on the document \"First-Time Calling of Qwen API\"..."},
        {"role": "user", "content": "How do I configure my API Key as an environment variable?"}
      ]
    },
    "parameters": {"agent_options": {"agent_id": "aid-xxxxxxxx"}},
    "stream": true
  }'
The tool_call_id must match exactly the value returned in the first turn’s streaming response—not generated manually. The model uses it to associate each tool result with its corresponding call.

6. Effect Comparison

Using the “First-Time Calling of Qwen API” document as a test case: After Turn 1 (“How do I call the Qwen API?”), Turn 2 asks “How do I configure my API Key as an environment variable?” Here's how two history-passing strategies compare:
Full Tool History PassedText-Only History Passed
Tool Calls in Turn 20 — direct answer using prior resultsRedundant retrieval (2 semantic_search calls observed)
Response SpeedFast — no retrieval overheadSlower — full retrieval re-executed
Answer ConsistencyConsistent — based on identical retrieved chunksInconsistent — new retrieval may yield different chunks
Real-world measurement: Turn 1 executed 2 tool calls (semantic_search → obtain_file) generating 6 history messages. When Turn 2 passed full history (7 messages), the model answered directly with 0 tool calls. With text-only history (3 messages), it triggered 2 new semantic_search calls. Exact round counts and message numbers depend on document structure and query matching—but the conclusion holds reliably: passing full history avoids redundant retrieval.
Passing full history increases input length (including tool outputs), but eliminates retrieval overhead. Text-only history yields shorter inputs yet incurs full retrieval latency. While trade-offs exist, passing full history delivers superior speed and consistency for multi-turn follow-ups.

Extracting Tool History from SSE Stream

Agent streaming uses Server-Sent Events (SSE). Each data frame is data:{...}, and the relevant path is obj.output.choices[0].message. The field message.extra.step_change marks state transitions—focus only on three event types:
EventIdentification ConditionData to Extract
Tool Callstep_change == "tool_calling"tool_calls array (all parallel calls in this round)
Tool Returnstep_change == "tool_return" and role == "tool"content + tool_call_id
Final AnswerBetween generation_start and generation_endAccumulate all content fragments

State Flow 

plan_start (reasoning begins) ──→ plan_end ──→ tool_calling (dispatch tool calls) ──→ tool_return (tools return results) ──→ Need more info?
    ▲                                                                                                                        │
    └──────────────────────────────────────── Yes ───────────────────────────────────────────────────────────────────────────┤
                                                                                                                             │ No
                                                                                                                             ▼
                                                                                                              generation_start ──→ generation_end
The Agent may perform multiple tool-call rounds until sufficient information is gathered before generating the final answer.

Message Format and Key Rules

Extracted history messages must be ordered by time and conform to these roles and fields:
RoleRequired FieldsDescription
userrole, contentUser input
assistant (tool call)role, content (empty), tool_callsAll parallel tool calls in this round; function.arguments is a JSON string
toolrole, content, tool_call_idTool result; tool_call_id must match tool_calls[].id
assistant (final answer)role, contentModel’s final answer
Messages must be strictly time-ordered: 
user query ──→ assistant (tool_calls) ──→ tool × N ──→ assistant (tool_calls) ──→ tool × N ──→ assistant final answer ──→ user next turn
Multiple tool-call rounds repeat the assistant → tool pattern. Each new turn starts with a fresh user message and repeats the full flow—including all prior history.

Complete Python Implementation

The code below implements the full workflow: extracting history from SSE, auto-passing it across turns, and comparing outcomes. It’s tested against the sample documents and prompts used in this guide.

Configuration 

import json
import requests

# Retrieved from the API Debugging page of your Q&A service in the console
API_URL = "https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat"
API_KEY = "sk-xxxxxxxx"      # Workspace-scoped API Key (starts with "sk-ws-")
AGENT_ID = "aid-xxxxxxxx"    # Knowledge Q&A service ID

Extract Tool History from SSE Stream 

class ConversationTracker:
    """Tracks a single Agent response, extracting full message sequence from streaming events."""

    def __init__(self):
        self.messages = []          # Extracted message sequence (tool calls + returns)
        self.tool_call_rounds = 0   # Number of tool-call rounds (for effect comparison)
        self.answer = ""            # Final answer
        self._generating = False

    def process_sse_line(self, line: str):
        if not line.startswith("data:"):
            return
        obj = json.loads(line[len("data:"):].strip())
        choices = obj.get("output", {}).get("choices", [])
        if not choices:
            return

        msg = choices[0].get("message", {})
        step_change = msg.get("extra", {}).get("step_change", "")
        role = msg.get("role", "")
        content = msg.get("content", "") or ""

        # Event 1: Tool call dispatch (step_change == "tool_calling") → extract tool_calls
        if step_change == "tool_calling" and msg.get("tool_calls"):
            self.tool_call_rounds += 1
            self.messages.append({
                "role": "assistant", "content": "",
                "tool_calls": [{"id": tc["id"], "type": "function",
                                "function": {"name": tc["function"]["name"],
                                             "arguments": tc["function"]["arguments"]}}
                               for tc in msg["tool_calls"]],
            })
        # Event 2: Tool return (step_change == "tool_return", role == "tool") → extract content + tool_call_id
        elif step_change == "tool_return" and role == "tool":
            self.messages.append({"role": "tool",
                                  "tool_call_id": msg.get("tool_call_id", ""),
                                  "content": content})
        # Event 3: Final answer (generation_start → content stream → generation_end) → accumulate content
        elif step_change == "generation_start":
            self._generating = True
            self.answer = content
        elif self._generating and content:
            self.answer += content
        elif step_change == "generation_end":
            self.answer += content
            self._generating = False

    def get_response_messages(self):
        """Returns the full extracted message sequence (including final answer)."""
        result = list(self.messages)
        if self.answer:
            result.append({"role": "assistant", "content": self.answer})
        return result

Multi-Turn Conversation Manager 

class MultiTurnChat:
    """Manages multi-turn history, automatically carrying the full tool-call chain."""

    def __init__(self):
        self.history = []

    def ask(self, question: str) -> str:
        """Sends a question, auto-appends full history (including tool chain), returns final answer."""
        self.history.append({"role": "user", "content": question})
        sent = len(self.history)
        tracker = self._call_agent(self.history)
        # Append this turn’s tool chain and final answer to history
        self.history.extend(tracker.get_response_messages())
        print(f"  Sent {sent} messages; {tracker.tool_call_rounds} tool-call rounds")
        return tracker.answer

    def _call_agent(self, messages):
        headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
        body = {
            "input": {"messages": messages},
            "parameters": {"agent_options": {"agent_id": AGENT_ID}},
            "stream": True,
        }
        tracker = ConversationTracker()
        resp = requests.post(API_URL, headers=headers, json=body, stream=True, timeout=180)
        for chunk in resp.iter_lines():
            if chunk:
                tracker.process_sse_line(chunk.decode("utf-8").strip())
        return tracker

Effect Comparison: Full History vs. Text-Only History 

chat = MultiTurnChat()

# Turn 1
print("[Turn 1] How do I call the Qwen API?")
chat.ask("How do I call the Qwen API?")

# Turn 2 · Full tool history (auto-carried)
print("\n[Turn 2 · Full tool history]")
chat_full = MultiTurnChat()
chat_full.history = list(chat.history)
chat_full.ask("How do I configure my API Key as an environment variable?")

# Turn 2 · Text-only history (drop tool calls, keep only user + assistant replies)
print("\n[Turn 2 · Text-only history]")
text_only = [m for m in chat.history
             if m["role"] == "user" or (m["role"] == "assistant" and not m.get("tool_calls"))]
chat_text = MultiTurnChat()
chat_text.history = text_only
chat_text.ask("How do I configure my API Key as an environment variable?")
Sample output: 
[Turn 1] How do I call the Qwen API?
  Sent 1 message; 2 tool-call rounds

[Turn 2 · Full tool history]
  Sent 7 messages; 0 tool-call rounds      ← Reused; no redundant retrieval

[Turn 2 · Text-only history]
  Sent 3 messages; 2 tool-call rounds      ← Redundant retrieval

Important Notes

Controlling History Length

Tool return content can be lengthy; cumulative history may exceed context limits. Recommended mitigation strategies:
  • Truncate older tool returns: Keep full tool content only for the most recent K rounds; truncate earlier tool messages to first N characters
  • Sliding window: For histories >3 turns, replace earlier tool results with summaries
def truncate_history(messages, max_tool_len=2000, keep_recent=2):
    """Truncates tool return content from earlier rounds."""
    user_indices = [i for i, m in enumerate(messages) if m["role"] == "user"]
    if len(user_indices) <= keep_recent:
        return messages
    cutoff = user_indices[-keep_recent]
    result = []
    for i, msg in enumerate(messages):
        if i < cutoff and msg["role"] == "tool" and len(msg["content"]) > max_tool_len:
            msg = {**msg, "content": msg["content"][:max_tool_len] + "\n...(truncated)"}
        result.append(msg)
    return result

Content Not Required in History

ContentReason
reasoning_contentInternal reasoning—useful for UI display, but unnecessary for re-injection
extra fieldStreaming protocol metadata—used only for client-side rendering
usageToken usage stats—relevant only for monitoring and billing
tool_call_chunksStreaming fragments—already merged into tool_calls

Common Questions

What happens if I omit tool call history? The model loses retrieval context and redundantly re-searches identical queries—degrading answer quality. Real-world test: full history enables zero tool calls on follow-up; omission triggers full re-retrieval. Can I skip intermediate tool calls and send only the final answer? It works functionally but is strongly discouraged. Sending only user + assistant(final answer) strips the model of memory about knowledge base content, significantly weakening follow-up performance. Does order matter for parallel tool returns? Order doesn’t affect functionality—the model matches results via tool_call_id. However, all tool messages must appear immediately after their corresponding assistant(tool_calls) message.
Agent Development
Agent Context
Agent Governance & Evolution
Store
Application Marketplace
Use Cases
Support