← Heatmap / Gemini 3.1 Flash Lite · Task 106
Gemini 3.1 Flash Lite · Tool use · Task 106 (Traps, missing gross pay)
Mean 90.0%
| Field | Expected | Rollout #1 (90%) | Rollout #2 (90%) | Rollout #3 (90%) |
|---|---|---|---|---|
| decision | clarify | clarify | clarify | clarify |
| reason code | MISSING_GROSS_PAY | MISSING_GROSS_PAY | MISSING_GROSS_PAY | MISSING_GROSS_PAY |
| clauses | R1 R7 | R1 R7 | R1 R7 | R1 R7 |
Prompt
Process this monthly payroll record under the supplied rulebook. Return the R6 or R7 JSON answer. Retrieve the case inputs with list_documents and read_document. Reference rows are available only through lookup_reference. Use calculate for exact decimal arithmetic.
Rollouts
Rollout #1 90.0% explanation · 4 tool calls · 5.5s · $0.0029
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R7 |
| exact | 100.0% | R1 R7 |
| field accuracy | 100.0% | R1 R7 |
| citations | 100.0% | R1 R7 |
| explanation | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "b256826a42768acd-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="b256826a42768acd-payroll")Tool
{
"kind": "payroll",
"record": {"payroll_month": "2026-04"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "5000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "2800.00",
"middle_rate": "0.12",
"upper_rate": "0.18"
}
Assistant
calculate(program="gross_pay = 0.00\nemployee_contribution = 0.00\nallowed_deduction = 0.00\ntaxable_pay = 0.00\nincome_tax = 0.00\nnet_pay = 0.00")Tool
{
"results": {"gross_pay": "0.00", "employee_contribution": "0.00", "allowed_deduction": "0.00", "taxable_pay": "0.00", "income_tax": "0.00", "net_pay": "0.00"}
}
Assistant
{
"decision": "clarify",
"reason_code": "MISSING_GROSS_PAY",
"clauses": ["R1", "R7"]
}
Rollout #2 90.0% explanation · 4 tool calls · 5.6s · $0.0032
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R7 |
| exact | 100.0% | R1 R7 |
| field accuracy | 100.0% | R1 R7 |
| citations | 100.0% | R1 R7 |
| explanation | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "b256826a42768acd-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="b256826a42768acd-payroll")Tool
{
"kind": "payroll",
"record": {"payroll_month": "2026-04"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "5000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "2800.00",
"middle_rate": "0.12",
"upper_rate": "0.18"
}
Assistant
calculate(program="gross_pay = 0.00\nemployee_contribution = round(min(gross_pay, 5000.00) * 0.07, 2)\nallowed_deduction = 0.00\ntaxable_pay = gross_pay - employee_contribution - allowed_deduction\ntax_bracket1 = max(0, min(taxable_pay, 1200.00) - 0.00) * 0.00\ntax_bracket2 = max(0, min(taxable_pay, 2800.00) - 1200.00) * 0.12\ntax_bracket3 = max(0, taxable_pay - 2800.00) * 0.18\nincome_tax = round(tax_bracket1 + tax_bracket2 + tax_bracket3, 2)\nnet_pay = gross_pay - employee_contribution - allowed_deduction - income_tax")Tool
{
"results": {"gross_pay": "0.00", "employee_contribution": "0.00", "allowed_deduction": "0.00", "taxable_pay": "0.00", "tax_bracket1": "0.00", "tax_bracket2": "0.00", "tax_bracket3": "0.00", "income_tax": "0.00", "net_pay": "0.00"}
}
Assistant
{
"decision": "clarify",
"reason_code": "MISSING_GROSS_PAY",
"clauses": ["R1", "R7"]
}
Rollout #3 90.0% explanation · 4 tool calls · 5.1s · $0.0029
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R7 |
| exact | 100.0% | R1 R7 |
| field accuracy | 100.0% | R1 R7 |
| citations | 100.0% | R1 R7 |
| explanation | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "b256826a42768acd-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="b256826a42768acd-payroll")Tool
{
"kind": "payroll",
"record": {"payroll_month": "2026-04"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "5000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "2800.00",
"middle_rate": "0.12",
"upper_rate": "0.18"
}
Assistant
calculate(program="gross_pay = 0.00\nemployee_contribution = 0.00\nallowed_deduction = 0.00\ntaxable_pay = 0.00\nincome_tax = 0.00\nnet_pay = 0.00")Tool
{
"results": {"gross_pay": "0.00", "employee_contribution": "0.00", "allowed_deduction": "0.00", "taxable_pay": "0.00", "income_tax": "0.00", "net_pay": "0.00"}
}
Assistant
{
"decision": "clarify",
"reason_code": "MISSING_GROSS_PAY",
"clauses": ["R1", "R7"]
}