← Heatmap / Gemini 3.1 Flash Lite · Task 88
Gemini 3.1 Flash Lite · Tool use · Task 88 (Traps, missing month)
Mean -3.3%
| Field | Expected | Rollout #1 (-50%) | Rollout #2 (-50%) | Rollout #3 (90%) |
|---|---|---|---|---|
| decision | clarify | calculate | calculate | clarify |
| currency | LCR | LCR | ||
| gross pay | 2924.24 | 2924.24 | ||
| employee contribution | 146.21 | 146.21 | ||
| allowed deduction | 307.83 | 307.83 | ||
| taxable pay | 2470.20 | 2470.20 | ||
| income tax | 101.62 | 101.62 | ||
| net pay | 2368.58 | 2368.58 | ||
| reason code | MISSING_PAYROLL_MONTH | MISSING_PAYROLL_MONTH | ||
| clauses | R1 R7 | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 | R1 R7 |
Prompt
Process this monthly payroll record under the supplied rulebook. Return the R6 or R7 JSON answer. Retrieve the case inputs with list_documents and read_document. Reference rows are available only through lookup_reference. Use calculate for exact decimal arithmetic.
Rollouts
Rollout #1 -50.0% answered trap · 4 tool calls · 4.3s · $0.0033
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R1 R7 |
| exact | 0.0% | R1 R7 |
| field accuracy | 0.0% | R1 R7 |
| citations | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "53bdb78b6184cd21-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="53bdb78b6184cd21-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "2924.24", "allowed_deduction": "307.83"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.05",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.08",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(2924.24, 4500.00) * 0.05, 2)\ntaxable_pay = 2924.24 - employee_contribution - 307.83\ntax_free_amount = 1200.00\nupper_boundary = 3000.00\nmiddle_rate = 0.08\nupper_rate = 0.24\ntax_middle = max(0, min(taxable_pay, upper_boundary) - tax_free_amount) * middle_rate\ntax_upper = max(0, taxable_pay - upper_boundary) * upper_rate\nincome_tax = round(tax_middle + tax_upper, 2)\nnet_pay = 2924.24 - employee_contribution - 307.83 - income_tax")Tool
{
"results": {"employee_contribution": "146.21", "taxable_pay": "2470.20", "tax_free_amount": "1200.00", "upper_boundary": "3000.00", "middle_rate": "0.08", "upper_rate": "0.24", "tax_middle": "101.6160", "tax_upper": "0.00", "income_tax": "101.62", "net_pay": "2368.58"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "2924.24",
"employee_contribution": "146.21",
"allowed_deduction": "307.83",
"taxable_pay": "2470.20",
"income_tax": "101.62",
"net_pay": "2368.58",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #2 -50.0% answered trap · 4 tool calls · 4.4s · $0.0032
| Component | Score | Clauses |
|---|---|---|
| decision | 0.0% | R1 R7 |
| exact | 0.0% | R1 R7 |
| field accuracy | 0.0% | R1 R7 |
| citations | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "53bdb78b6184cd21-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="53bdb78b6184cd21-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "2924.24", "allowed_deduction": "307.83"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.05",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.08",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(2924.24, 4500.00) * 0.05, 2)\ntaxable_pay = 2924.24 - employee_contribution - 307.83\ntax_bracket1 = max(0, min(taxable_pay, 3000.00) - 1200.00)\ntax_bracket2 = max(0, taxable_pay - 3000.00)\nincome_tax = round(tax_bracket1 * 0.08 + tax_bracket2 * 0.24, 2)\nnet_pay = 2924.24 - employee_contribution - 307.83 - income_tax")Tool
{
"results": {"employee_contribution": "146.21", "taxable_pay": "2470.20", "tax_bracket1": "1270.20", "tax_bracket2": "0", "income_tax": "101.62", "net_pay": "2368.58"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "2924.24",
"employee_contribution": "146.21",
"allowed_deduction": "307.83",
"taxable_pay": "2470.20",
"income_tax": "101.62",
"net_pay": "2368.58",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #3 90.0% explanation · 4 tool calls · 4.1s · $0.0032
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R7 |
| exact | 100.0% | R1 R7 |
| field accuracy | 100.0% | R1 R7 |
| citations | 100.0% | R1 R7 |
| explanation | 0.0% | R1 R7 |
Assistant
list_documents()Tool
[
{"id": "53bdb78b6184cd21-payroll", "kind": "payroll"}
]
Assistant
read_document(document_id="53bdb78b6184cd21-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "2924.24", "allowed_deduction": "307.83"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.05",
"cap": "4500.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.08",
"upper_rate": "0.24"
}
Assistant
calculate(program="employee_contribution = round(min(2924.24, 4500.00) * 0.05, 2)\ntaxable_pay = round(2924.24 - employee_contribution - 307.83, 2)\ntax_bracket_1 = max(0, min(taxable_pay, 1200.00) - 0)\ntax_bracket_2 = max(0, min(taxable_pay, 3000.00) - 1200.00)\ntax_bracket_3 = max(0, taxable_pay - 3000.00)\nincome_tax = round(tax_bracket_1 * 0.00 + tax_bracket_2 * 0.08 + tax_bracket_3 * 0.24, 2)\nnet_pay = round(2924.24 - employee_contribution - 307.83 - income_tax, 2)")Tool
{
"results": {"employee_contribution": "146.21", "taxable_pay": "2470.20", "tax_bracket_1": "1200.00", "tax_bracket_2": "1270.20", "tax_bracket_3": "0", "income_tax": "101.62", "net_pay": "2368.58"}
}
Assistant
{
"decision": "clarify",
"reason_code": "MISSING_PAYROLL_MONTH",
"clauses": ["R1", "R7"]
}