Baseline evaluation provides 13 public standard benchmarks across five major categories: general knowledge, mathematics, code, context reasoning, and domain-specific. It requires no dataset preparation or evaluation dimension configuration on your part and quickly delivers benchmark scores for model basic capabilities.
Create a baseline evaluation task overview
Baseline evaluation provides 13 public standard benchmarks across five major categories: general knowledge, mathematics, code, context reasoning, and domain-specific. It automatically evaluates model basic capabilities—no need to prepare datasets or configure evaluation dimensions yourself—so you can quickly obtain benchmark scores.
Benchmark library
The benchmark library is organized into 5 major categories:
- General knowledge: Assesses model performance in general capabilities such as multidisciplinary expertise, multilingual understanding, and instruction following.
- Mathematics: Assesses the model's mathematical reasoning and competition mathematics capabilities.
- Code: Assesses the model's code generation capabilities.
- Context reasoning: Assesses the model's retrieval, understanding, and reasoning capabilities in long texts.
- Domain-specific: Assesses the model's capabilities in professional scenarios such as finance and medicine.
Category | Benchmark | Tags | Questions | Evaluation dimensions | Description |
|---|---|---|---|---|---|
General knowledge | MMLU-Pro | Knowledge QA, 10 options | 12,032 | 14 items | Enhanced multiple-choice questions covering 14 disciplines such as STEM, testing professional knowledge and deep reasoning capabilities |
General knowledge | MMMLU | Text reasoning, multilingual | 196,588 | 14 items | The multilingual version of MMLU, with test questions professionally translated into 14 languages to assess multilingual capabilities |
General knowledge | BrowseComp-ZH-Revised | Reasoning, text, Chinese | 289 | — | A high-difficulty benchmark for the Chinese web environment, with 289 multi-hop questions across 11 domains |
General knowledge | IFEval | Instruction following, 25 rule types | 541 | — | Covers 25 types of verifiable instruction rules such as format constraints and length limits, assessing the capability to precisely follow instructions |
Mathematics | GSM8K | Mathematics, text, English | 1,319 | — | Elementary school math word problems that require multi-step reasoning and basic arithmetic, assessing fundamental mathematical reasoning capabilities |
Mathematics | MATH-500 | Mathematical reasoning, 5 difficulty levels | 500 | 5 items | 500 graded competition math problems covering algebra, geometry, and number theory |
Mathematics | AIME-2025 | Competition mathematics, integer answers | 30 | — | 30 high-difficulty competition problems from the American Invitational Mathematics Examination |
Code | HumanEval | Python function generation | 164 | — | 164 Python function generation problems (including signatures, docstrings, and test cases), testing code generation capabilities |
Context reasoning | LongBench-v2 | Bilingual, 6 task types, long text | 503 | 3 items | Covers 6 types of bilingual long-text tasks, including single/multi-document QA, long-context learning, and code repository understanding |
Context reasoning | OpenAI-MRCR | Long context, coreference resolution, bilingual | 2,400 | 3 items | Multi-turn dialogue coreference resolution tasks, testing the capability to precisely retrieve and distinguish similar content in ultra-long contexts |
Domain-specific | FinanceQA | Financial reasoning, numerical calculation, multi-step | 148 | 3 items | Multi-step numerical calculation problems from real financial analysis scenarios such as hedge funds and investment banks, testing financial reasoning capabilities |
Domain-specific | PubMedQA | Biomedical, three-class, literature QA | 1,000 | — | Research QA based on PubMed literature abstracts, with yes/no/maybe three-class answers |
Domain-specific | MedMCQA | Medicine, 21 disciplines, MCQ | 4,183 | — | Multiple-choice questions covering 21 medical disciplines such as anatomy, physiology, and pharmacology |
Creation workflow
- In the Model Studio console, select the Evaluation Tasks tab and click Create Evaluation Task.
- Select Baseline Evaluation as the evaluation method.
- Select the target model to evaluate. You can select Official Models (Qwen-series commercial and open-source models) and My Models (deployed fine-tuned models).
- Select baseline evaluation data. The benchmark library is displayed in 5 category tabs, with cards containing the benchmark name, tags, description, and number of questions, and multiple selection is supported. Some benchmarks support Evaluation Dimension selection, with all selected by default; you can deselect to run only the specified sub-dimension data.
- (Optional) Enable the Data Sampling Percentage switch to sample evaluation data by percentage and shorten the evaluation duration; when disabled, all data is evaluated.
- Click Start Evaluation to submit the task. Costs are calculated based on actual inference and scoring consumption, and you can view the calculation details before submission.
Differences from custom evaluation
- Baseline evaluation does not use custom evaluation dimensions. The system automatically scores with preset standards.
- Baseline evaluation does not support inference result sets as the data source and only uses platform preset public datasets.
- Baseline evaluation tasks display the overall score in the task list, without the dimension details column (a dash is displayed).
- Baseline evaluation tasks cannot enter the manual annotation page.
FAQ
Question: I cannot see the baseline evaluation option when creating an evaluation task.
Question: I cannot see the baseline evaluation option when creating an evaluation task.