The Agent versioning mechanism, evaluation with a labeled test set, session version-locking semantics, and canary/rollback practice — turning the prompt into a server-side versionable resource.
Overview
This page covers the Managed Agents Agent versioning mechanism, version canary and rollback practice, with REST/curl examples.
The core idea: strip the prompt out of application code and make it a server-side versionable resource that goes through review and promotion independently of code. The mechanisms involved: POST /agents/{id} (each update bumps the version), session-to-agent version binding (explicitly locking a historical version is supported), and GET /agents/{id}/versions to list history.
Scenario
A product support system routes tickets to the right team with an LLM. The PM wants more API-related tickets flowing to the platform team.
Previously, changing one routing-prompt line meant "PR + CI + deploy", and rollback meant the same pipeline. With Managed Agents, the prompt lives server-side: each POST /agents/{id} produces a new, immutable version; a session locks an agent version snapshot at creation (latest by default, or explicitly specified). If the new version regresses, rollback is "publish a new version equivalent to the old one" — newly created sessions use it immediately, no deployment needed.
After this page you can:
- create an Agent that returns
version: 1; - score any version against a labeled test set;
- publish v2 and watch the version auto-increment;
- evaluate a historical version without touching the current one, by locking it explicitly;
- pull a regressed Agent back to v1 behavior with zero deploys.
Environment variables
The curl examples use:
Authorization: Bearer $DASHSCOPE_API_KEY — one key covers all workspace resources. The region is currently cn-beijing only. Use model IDs qwen3.8-max / qwen3.7-max / qwen3.7-plus (this page uses qwen3.7-plus, plenty for classification).
The Agent here is a pure LLM classifier (no tool calls) and needs no Environment — a session without
environment_id runs fine. Only agents that run bash or read/write files need a sandbox; see Getting Started: Build a Data Analysis Agent.Step 1: Create the Agent (v1)
The system prompt has the Agent classify each ticket into "team + priority" and answer with a single JSON line:
version: 1. Every later POST /agents/{id} (the update verb is POST, not PATCH — PATCH returns 405; the body requires both name and the current version) produces a new version under the same id with an auto-incremented number — that number is the handle for binding and rollback below. Fill the response id into AGENT_ID.
Step 2: Load the labeled test set
The fixture is these 4 tickets (one per team; T3 is deliberately an "invoice problem that mentions API usage" to set up the regression):
Step 3: Score v1 (establish the baseline)
Run v1 once per ticket to get the baseline. Each ticket's evaluation (triage) goes:
- create a session (
agentas the agent ID); - post a
messageevent carrying the ticket; - poll history until the session returns to idle;
- collect the
assistantoutput, parse the JSON, compare against ground truth; - delete the session (evaluated and gone, no residue).
Session ↔ agent version binding semantics (important)
The agent field has two forms, deciding which version the session locks:
agent form | Locked version |
|---|---|
"agent_xxx" (string) | Latest at the moment of creation |
{ "id": "agent_xxx", "version": N } (object) | Explicitly version N; a missing version 404s ("Agent not found") |
{ "id": "agent_xxx" } (object without version) | Same as the string form — latest |
- The session uses that snapshot for its whole life; later agent upgrades do not affect an existing session.
- A top-level
versionfield (sibling ofagent) is ignored — it still locks latest.
- To evaluate any historical version, pass the object form
{id, version: N}— no need to make it "current" first (counter-intuitively, many assume you must roll back before evaluating history; you don't). - Default production traffic uses the string form and locks latest, so "which version does production run" = "who controls the agent's latest state".
triage over REST
Create the session (string form locks the current latest snapshot; a pure-LLM agent needs no environment_id):
Session creation takes no initial input field (no
initial_events) — the pattern is "create the session, then post one message event".SESSION_ID from the response, send the ticket (subject and body concatenated into the text):
Parsing detail: the
session_status / stop_reason you check for completion live in the session_status-typed event's content[0].data ({"stop_reason": {"type": "end_turn"}, "session_status": "idle"}), not at the top level; a top-level type of session_status marks turn end. Set a 60-second timeout. Each triage takes roughly 8–11 seconds.content text of the assistant message, json.loads the {"team", "priority"}, compare with the expected team; then delete the session:
GET /sessions/{id}/events/stream with Accept: text/event-stream), accumulating text by sequence_number; polling is simpler in an eval script. Archiving (POST /sessions/{id}/archive) and deleting are interchangeable; DELETE is cleaner inside an eval loop.
The score logic: run one triage per ticket and tally hits/total by team.
v1 baseline (4-ticket set, qwen3.7-plus):
| Team | Hit/total |
|---|---|
| api-platform | 1/1 |
| auth | 1/1 |
| billing | 1/1 (T3 mentions API usage and still routes correctly to billing) |
| dashboard | 1/1 |
Step 4: Publish v2
The PM adds a routing rule: any ticket mentioning API usage, rate limits, quotas, or request volume goes to the platform team. Update the system prompt with POST /agents/{id} (name + current version required) — that produces v2.
The new prompt = v1 plus:
version becomes 2.
Interlude: where did code review go?
One API call changed the agent with no review in between — fine for a demo, not for production. POST /agents/{id} has no built-in approval flow; any key in the workspace can call it. And string-form sessions lock the latest at creation — the first session created after an upgrade uses the new prompt. This is exactly the situation with feature flags or any "config managed via API rather than code".
Recommended pattern to restore review:
- make "which version production runs" a change-controlled artifact — e.g. explicitly record the target version in your deployment config;
- anyone can create v2 / v3 / v10; those versions just sit server-side with no production traffic;
- "promotion" = updating that "tell production which version" config, which goes through normal code review. Advanced: have production create sessions with the
{id, version}object locked to the controlled number, not even depending on "the agent's latest version".
Step 5: Score v2
Run the same eval against v2. Two options:
- Default: v2 is now latest — plain string-form sessions (locking v2);
- Comparative: leave the agent alone — object-form sessions locking
{id, version: 1}and{id, version: 2}each, comparing any two historical versions at will.
| Team | Hit/total |
|---|---|
| api-platform | 1/1 |
| auth | 1/1 |
| billing | 0/1 (regression! T3 routed to api-platform) |
| dashboard | 1/1 |
Step 6: Rollback
billing regressed. v1's configuration is fully intact server-side — rollback needs no code deployment.
Rollback = publish a new version equivalent to v1. Default traffic (string form) binds "latest at creation", so to restore v1 behavior, POST v1's system prompt verbatim once more with POST /agents/{id} — producing v3 (the number keeps incrementing; the configuration equals v1):
GET /agents/{id}/versions (below).
Re-run the failing T3 against the v1-equivalent v3: billing routes correctly again.
Since every version persists server-side:
- v2 remains available for iteration (publish v4 based on v2, or lock-evaluate
{id, version: 2}); - production can stay on v3, or canary — a fraction of requests passing
{id, version: N}on the new version, the rest defaulting to latest; - once fixed, publish a new version and repeat the eval → promotion flow.
Cleanup
Archive rather than delete: agents do not support DELETE (405). Archiving keeps the audit trail, tears down the container, stops metering — version history is the point of the resource. The eval sessions were already deleted in the loop.
Summary
The mechanism is small — create versions, update to bump, lock production to a version, switch back when needed — but once the prompt becomes a versioned server-side resource, it can be evaluated and promoted independently of application code. Key points:
- "Which version production runs" should be a controlled artifact: treat it as the gate for prompt changes and route it through normal review.
- Canary high-risk agents before full rollout (split traffic with
{id, version}locking). - Understand the binding semantics: string form locks latest; object form
{id, version}locks a specific version (404 if absent); a top-levelversionfield is ignored; existing sessions are unaffected by later upgrades. - Evaluating history needs no rollback — the object form locks any version for comparison; rollback exists to restore default traffic to old behavior.
Appendix: list all versions
config is a raw dump of the internal storage format: fields are camelCase (mcpServers, workspaceId, gmtCreate), unlike the creation API's snake_case; each version carries the full system and created_at, usable for audit, historical comparison, and deciding which version to promote.
Next steps
- Advanced: Multi-Agent Custom Proposals: the roster member version-locking semantics are dual to session locking.
- Advanced: Productionize a Morning-Briefing Robot: deployment version locking and the drift trap.
- Define an Agent: versions and update semantics from the console's perspective.