← Heatmap / Gemini 3.1 Flash Lite · Task 28
Gemini 3.1 Flash Lite · Tool use · Task 28 (Traps, missing month)
Mean -50.0%
| Field | Expected | Rollout #1 (-50%) | Rollout #2 (-50%) | Rollout #3 (-50%) |
|---|---|---|---|---|
| decision | clarify | calculate | calculate | calculate |
| currency | LCR | LCR | LCR | |
| gross pay | 1930.34 | 1930.34 | 1930.34 | |
| employee contribution | 135.12 | 135.12 | 135.12 | |
| allowed deduction | 228.05 | 228.05 | 228.05 | |
| taxable pay | 1567.17 | 1567.17 | 1567.17 | |
| income tax | 66.72 | 66.72 | 66.72 | |
| net pay | 1500.45 | 1500.45 | 1500.45 | |
| reason code | MISSING_PAYROLL_MONTH | |||
| clauses | R1 R7 | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 |
Prompt
Process this monthly payroll record under the supplied rulebook. Return the R6 or R7 JSON answer. Retrieve the case inputs with list_documents and read_document. Reference rows are available only through lookup_reference. Use calculate for exact decimal arithmetic.
Rollouts
Rollout #1 -50.0% answered trap · 4 tool calls · 6.1s · $0.0033
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R1 R7 |
| exact | 0.0% | R1 R7 |
| field accuracy | 0.0% | R1 R7 |
| citations | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "5907abef11c974cf-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="5907abef11c974cf-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1930.34", "allowed_deduction": "228.05"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "4000.00",
"deduction": "400.00",
"free": "900.00",
"upper": "2800.00",
"middle_rate": "0.10",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(1930.34, 4000.00) * 0.07, 2)\ntaxable_pay = 1930.34 - employee_contribution - 228.05\n# Income tax:\n# 0-900: 0\n# 900-2800: 0.10\n# >2800: 0.24\n# Taxable pay = 1930.34 - 135.12 - 228.05 = 1567.17\n# Tax = (1567.17 - 900) * 0.10 = 66.717\nincome_tax = round(66.717, 2)\nnet_pay = 1930.34 - employee_contribution - 228.05 - income_tax\n")Tool
{
"results": {"employee_contribution": "135.12", "taxable_pay": "1567.17", "income_tax": "66.72", "net_pay": "1500.45"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1930.34",
"employee_contribution": "135.12",
"allowed_deduction": "228.05",
"taxable_pay": "1567.17",
"income_tax": "66.72",
"net_pay": "1500.45",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #2 -50.0% answered trap · 4 tool calls · 5.7s · $0.0033
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R1 R7 |
| exact | 0.0% | R1 R7 |
| field accuracy | 0.0% | R1 R7 |
| citations | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "5907abef11c974cf-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="5907abef11c974cf-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1930.34", "allowed_deduction": "228.05"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "4000.00",
"deduction": "400.00",
"free": "900.00",
"upper": "2800.00",
"middle_rate": "0.10",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(1930.34, 4000.00) * 0.07, 2)\ntaxable_pay = round(1930.34 - employee_contribution - 228.05, 2)\ntax_bracket_1 = max(0, min(taxable_pay, 900.00) - 0)\ntax_bracket_2 = max(0, min(taxable_pay, 2800.00) - 900.00)\ntax_bracket_3 = max(0, taxable_pay - 2800.00)\nincome_tax = round(tax_bracket_1 * 0 + tax_bracket_2 * 0.10 + tax_bracket_3 * 0.24, 2)\nnet_pay = round(1930.34 - employee_contribution - 228.05 - income_tax, 2)")Tool
{
"results": {"employee_contribution": "135.12", "taxable_pay": "1567.17", "tax_bracket_1": "900.00", "tax_bracket_2": "667.17", "tax_bracket_3": "0", "income_tax": "66.72", "net_pay": "1500.45"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1930.34",
"employee_contribution": "135.12",
"allowed_deduction": "228.05",
"taxable_pay": "1567.17",
"income_tax": "66.72",
"net_pay": "1500.45",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #3 -50.0% answered trap · 4 tool calls · 6.2s · $0.0033
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R1 R7 |
| exact | 0.0% | R1 R7 |
| field accuracy | 0.0% | R1 R7 |
| citations | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "5907abef11c974cf-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="5907abef11c974cf-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1930.34", "allowed_deduction": "228.05"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.07",
"cap": "4000.00",
"deduction": "400.00",
"free": "900.00",
"upper": "2800.00",
"middle_rate": "0.10",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(1930.34, 4000.00) * 0.07, 2)\ntaxable_pay = round(1930.34 - employee_contribution - 228.05, 2)\ntax_bracket_1 = max(0, min(taxable_pay, 900.00) - 0)\ntax_bracket_2 = max(0, min(taxable_pay, 2800.00) - 900.00)\ntax_bracket_3 = max(0, taxable_pay - 2800.00)\nincome_tax = round(tax_bracket_1 * 0.0 + tax_bracket_2 * 0.10 + tax_bracket_3 * 0.24, 2)\nnet_pay = round(1930.34 - employee_contribution - 228.05 - income_tax, 2)")Tool
{
"results": {"employee_contribution": "135.12", "taxable_pay": "1567.17", "tax_bracket_1": "900.00", "tax_bracket_2": "667.17", "tax_bracket_3": "0", "income_tax": "66.72", "net_pay": "1500.45"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1930.34",
"employee_contribution": "135.12",
"allowed_deduction": "228.05",
"taxable_pay": "1567.17",
"income_tax": "66.72",
"net_pay": "1500.45",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}