← Heatmap / MiMo V2.6 Pro · Task 42
MiMo V2.6 Pro · Tool use · Task 42 (Traps, conflicting pay)
Mean 50.0%
| Field | Expected | Rollout #1 (-50%) | Rollout #2 (100%) | Rollout #3 (100%) |
|---|---|---|---|---|
| decision | clarify | (malformed) | clarify | clarify |
| reason code | CONFLICTING_GROSS_PAY | (malformed) | CONFLICTING_GROSS_PAY | CONFLICTING_GROSS_PAY |
| clauses | R7 R9 | (malformed) | R7 R9 | R7 R9 |
Prompt
Process this monthly payroll record under the supplied rulebook. Return the R6 or R7 JSON answer. Retrieve the case inputs with list_documents and read_document. Reference rows are available only through lookup_reference. Use calculate for exact decimal arithmetic.
Rollouts
Rollout #1 -50.0% malformed · 4 tool calls · 12.0s · $0.0009
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R7 R9 |
| exact | 0.0% | R7 R9 |
| field accuracy | 0.0% | R7 R9 |
| citations | 0.0% | R7 R9 |
Assistant
list_documents()Reasoning
Let me start by listing documents.
Tool
[
{"id": "7b56e045b2787948-payroll", "kind": "payroll"},
{"id": "7b56e045b2787948-contract", "kind": "contract"}
]
Assistant
read_document(document_id="7b56e045b2787948-payroll")read_document(document_id="7b56e045b2787948-contract")Reasoning
Read both documents.
Tool
{
"kind": "payroll",
"record": {"gross_pay": "9728.06", "payroll_month": "2026-02"}
}
Tool
{
"kind": "contract",
"gross_pay": "9703.06"
}
Assistant
lookup_reference(reference="first-half")Reasoning
Payroll gross_pay = 9728.06, contract = 9703.06. They differ. No amendment document. So conflict → CONFLICTING_GROSS_PAY under R7 and R9. Wait — is there an amendment? Only two documents listed. No amendment. So clarify. But wait, careful: the conflict check order: missing gross_pay, missing payroll_month, no reference row, then conflicting gross_pay. Month 2026-02 is within 2026 and has a reference row (first-half covers months 1-6 presumably). Should I check reference rows first? The order says check in this order: 1 missing gross, 2 missing month, 3 month without reference row, 4 conflicting. 2026-02 should have a reference row (first-half). But should I verify with lookup_reference? The conflict applies anyway after. But to be thorough, check that the month has a reference row — if it didn't, NO_APPLICABLE_REFERENCE would take precedence. Let me look up first-half to confirm coverage. Actually, the row is needed only if calculating, but for ordering I should confirm the row exists. Let me lookup first-half.
Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.06",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3600.00",
"middle_rate": "0.08",
"upper_rate": "0.20"
}
Assistant
{"decision": "clarify", "reason_code": "CONFLICTING_GROSS_PAY", "clauses": ["R7", "R9"], "explanation": "The payroll record gross_pay 9728.06 differs from the contract gross_pay 9703.06, and no approved amendment resolves the conflict. Please resolve the conflicting gross pay amounts (e.g., by supplying an approved amendment matching the payroll record)."}```
Reasoning
Month 2026-02 is covered (2026-01 through 2026-06 presumably). Then check 4: contract_gross_pay 9703.06 ≠ gross_pay 9728.06, no amendment → CONFLICTING_GROSS_PAY, cite R7 and R9. No monetary fields.
Rollout #2 100.0% passed · 4 tool calls · 15.8s · $0.0009
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R7 R9 |
| exact | 100.0% | R7 R9 |
| field accuracy | 100.0% | R7 R9 |
| citations | 100.0% | R7 R9 |
| explanation | 100.0% | R7 R9 |
Assistant
list_documents()Reasoning
Let me start by listing documents.
Tool
[
{"id": "7b56e045b2787948-payroll", "kind": "payroll"},
{"id": "7b56e045b2787948-contract", "kind": "contract"}
]
Assistant
read_document(document_id="7b56e045b2787948-payroll")read_document(document_id="7b56e045b2787948-contract")Reasoning
Read both documents.
Tool
{
"kind": "payroll",
"record": {"gross_pay": "9728.06", "payroll_month": "2026-02"}
}
Tool
{
"kind": "contract",
"gross_pay": "9703.06"
}
Assistant
lookup_reference(reference="first-half")Reasoning
Payroll gross: 9728.06. Contract gross: 9703.06. They differ. No amendment provided. Under R7 #4 and R9, conflicting gross pay → clarify with CONFLICTING_GROSS_PAY, citing R7 and R9. But wait — R7 check order: gross_pay present, payroll_month present (2026-02 covered), no reference row? Let me check the reference row first? Actually the conflict check order is after the reference row check. Let me lookup the reference to confirm 2026-02 has a row (first-half). But since the conflict is present, the answer is clarify regardless — but order matters: check 3 before 4. Let me lookup reference first to be sure.
Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.06",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3600.00",
"middle_rate": "0.08",
"upper_rate": "0.20"
}
Assistant
{
"decision": "clarify",
"reason_code": "CONFLICTING_GROSS_PAY",
"clauses": ["R7", "R9"],
"explanation": "The payroll record shows gross_pay 9728.06 while the contract shows 9703.06, and no approved amendment resolves the difference. Please resolve the conflicting gross pay (e.g., by supplying an approved amendment matching the payroll record's gross pay and effective no later than 2026-02, or confirming the correct gross pay)."
}
Reasoning
The first-half row covers 2026-01 through 2026-06, so 2026-02 has a reference row. Check 3 passes. Check 4: contract_gross_pay 9703.06 differs from gross_pay 9728.06, no amendment → CONFLICTING_GROSS_PAY, cite R7 and R9.
Rollout #3 100.0% passed · 4 tool calls · 12.8s · $0.0009
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R7 R9 |
| exact | 100.0% | R7 R9 |
| field accuracy | 100.0% | R7 R9 |
| citations | 100.0% | R7 R9 |
| explanation | 100.0% | R7 R9 |
Assistant
list_documents()Reasoning
Let me list documents first.
Tool
[
{"id": "7b56e045b2787948-payroll", "kind": "payroll"},
{"id": "7b56e045b2787948-contract", "kind": "contract"}
]
Assistant
read_document(document_id="7b56e045b2787948-payroll")read_document(document_id="7b56e045b2787948-contract")Reasoning
Read both documents.
Tool
{
"kind": "payroll",
"record": {"gross_pay": "9728.06", "payroll_month": "2026-02"}
}
Tool
{
"kind": "contract",
"gross_pay": "9703.06"
}
Assistant
lookup_reference(reference="first-half")Reasoning
Payroll gross pay: 9728.06, contract gross pay: 9703.06. They differ. No amendment document provided. So conflict → CONFLICTING_GROSS_PAY, cite R7 and R9. Wait, but should I still check reference? Order of checks: gross_pay present, payroll_month present, reference row exists (2026-02 → first-half exists presumably), then conflict. Let me check reference row exists to ensure ordering. Actually R7 order: 1 missing gross, 2 missing month, 3 no reference row, 4 conflict. So I should verify reference row exists first (to determine order). If no reference row, reason would be NO_APPLICABLE_REFERENCE. Let me lookup first-half.
Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.06",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3600.00",
"middle_rate": "0.08",
"upper_rate": "0.20"
}
Assistant
{
"decision": "clarify",
"reason_code": "CONFLICTING_GROSS_PAY",
"clauses": ["R7", "R9"],
"explanation": "The payroll record gross_pay (9728.06 LCR) differs from the contract gross_pay (9703.06 LCR), and no approved amendment resolving the difference is on file. Please provide an approved amendment effective no later than 2026-02 matching the payroll gross pay, or confirm the correct gross pay."
}
Reasoning
2026-02 is covered by first-half. So reference row exists. Now conflict: gross_pay 9728.06 vs contract 9703.06 differ, no amendment → CONFLICTING_GROSS_PAY, cite R7 and R9.