← Heatmap / Gemini 3.1 Flash Lite · Task 17
Gemini 3.1 Flash Lite · Tool use · Task 17 (Calculation, at the tax-free line)
Mean 100.0%
| Field | Expected | Rollout #1 (100%) | Rollout #2 (100%) | Rollout #3 (100%) |
|---|---|---|---|---|
| decision | calculate | calculate | calculate | calculate |
| currency | LCR | LCR | LCR | LCR |
| gross pay | 1430.25 | 1430.25 | 1430.25 | 1430.25 |
| employee contribution | 114.42 | 114.42 | 114.42 | 114.42 |
| allowed deduction | 115.82 | 115.82 | 115.82 | 115.82 |
| taxable pay | 1200.01 | 1200.01 | 1200.01 | 1200.01 |
| income tax | 0.00 | 0.00 | 0.00 | 0.00 |
| net pay | 1200.01 | 1200.01 | 1200.01 | 1200.01 |
| clauses | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 | R1 R2 R3 R4 R5 R6 |
Prompt
Process this monthly payroll record under the supplied rulebook. Return the R6 or R7 JSON answer. Retrieve the case inputs with list_documents and read_document. Reference rows are available only through lookup_reference. Use calculate for exact decimal arithmetic.
Rollouts
Rollout #1 100.0% passed · 4 tool calls · 4.9s · $0.0034
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R2 R3 R4 R5 R6 |
| exact | 100.0% | R1 R2 R3 R4 R5 R6 |
| field accuracy | 100.0% | R1 R2 R3 R4 R5 R6 |
| citations | 100.0% | R1 R2 R3 R4 R5 R6 |
Assistant
list_documents()Tool
[
{"id": "c0b6cf337e756b28-payroll", "kind": "payroll"},
{"id": "c0b6cf337e756b28-contract", "kind": "contract"}
]
Assistant
read_document(document_id="c0b6cf337e756b28-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1430.25", "payroll_month": "2026-03", "allowed_deduction": "115.82"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.08",
"cap": "4000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.14",
"upper_rate": "0.18"
}
Assistant
calculate(program="employee_contribution = round(min(1430.25, 4000.00) * 0.08, 2)\ntaxable_pay = 1430.25 - employee_contribution - 115.82\ntax_free_portion = min(taxable_pay, 1200.00)\ntax_middle_portion = min(max(taxable_pay - 1200.00, 0), 3000.00 - 1200.00)\ntax_upper_portion = max(taxable_pay - 3000.00, 0)\nincome_tax = round(tax_middle_portion * 0.14 + tax_upper_portion * 0.18, 2)\nnet_pay = 1430.25 - employee_contribution - 115.82 - income_tax")Tool
{
"results": {"employee_contribution": "114.42", "taxable_pay": "1200.01", "tax_free_portion": "1200.00", "tax_middle_portion": "0.01", "tax_upper_portion": "0", "income_tax": "0.00", "net_pay": "1200.01"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1430.25",
"employee_contribution": "114.42",
"allowed_deduction": "115.82",
"taxable_pay": "1200.01",
"income_tax": "0.00",
"net_pay": "1200.01",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #2 100.0% passed · 5 tool calls · 6.5s · $0.0044
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R2 R3 R4 R5 R6 |
| exact | 100.0% | R1 R2 R3 R4 R5 R6 |
| field accuracy | 100.0% | R1 R2 R3 R4 R5 R6 |
| citations | 100.0% | R1 R2 R3 R4 R5 R6 |
Assistant
list_documents()Tool
[
{"id": "c0b6cf337e756b28-payroll", "kind": "payroll"},
{"id": "c0b6cf337e756b28-contract", "kind": "contract"}
]
Assistant
read_document(document_id="c0b6cf337e756b28-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1430.25", "payroll_month": "2026-03", "allowed_deduction": "115.82"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.08",
"cap": "4000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.14",
"upper_rate": "0.18"
}
Assistant
calculate(program="employee_contribution = round(min(1430.25, 4000.00) * 0.08, 2)\ntaxable_pay = 1430.25 - employee_contribution - 115.82\ntax_free_portion = min(taxable_pay, 1200.00)\ntaxable_above_free = max(0, taxable_pay - 1200.00)\nmiddle_tax = min(taxable_above_free, 3000.00 - 1200.00) * 0.14\nupper_tax = max(0, taxable_above_free - 1800.00) * 0.18\nincome_tax = round(middle_tax + upper_tax, 2)\nnet_pay = 1430.25 - employee_contribution - 115.82 - income_tax")Tool
{
"results": {"employee_contribution": "114.42", "taxable_pay": "1200.01", "tax_free_portion": "1200.00", "taxable_above_free": "0.01", "middle_tax": "0.0014", "upper_tax": "0.00", "income_tax": "0.00", "net_pay": "1200.01"}
}
Assistant
calculate(program="employee_contribution = round(min(1430.25, 4000.00) * 0.08, 2)\ntaxable_pay = 1430.25 - employee_contribution - 115.82\nmiddle_bracket = max(0, min(taxable_pay, 3000.00) - 1200.00)\nupper_bracket = max(0, taxable_pay - 3000.00)\nincome_tax = round(middle_bracket * 0.14 + upper_bracket * 0.18, 2)\nnet_pay = 1430.25 - employee_contribution - 115.82 - income_tax")Tool
{
"results": {"employee_contribution": "114.42", "taxable_pay": "1200.01", "middle_bracket": "0.01", "upper_bracket": "0", "income_tax": "0.00", "net_pay": "1200.01"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1430.25",
"employee_contribution": "114.42",
"allowed_deduction": "115.82",
"taxable_pay": "1200.01",
"income_tax": "0.00",
"net_pay": "1200.01",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}
Rollout #3 100.0% passed · 4 tool calls · 5.4s · $0.0034
| Component | Score | Clauses |
|---|---|---|
| decision | 100.0% | R1 R2 R3 R4 R5 R6 |
| exact | 100.0% | R1 R2 R3 R4 R5 R6 |
| field accuracy | 100.0% | R1 R2 R3 R4 R5 R6 |
| citations | 100.0% | R1 R2 R3 R4 R5 R6 |
Assistant
list_documents()Tool
[
{"id": "c0b6cf337e756b28-payroll", "kind": "payroll"},
{"id": "c0b6cf337e756b28-contract", "kind": "contract"}
]
Assistant
read_document(document_id="c0b6cf337e756b28-payroll")Tool
{
"kind": "payroll",
"record": {"gross_pay": "1430.25", "payroll_month": "2026-03", "allowed_deduction": "115.82"}
}
Assistant
lookup_reference(reference="first-half")Tool
{
"reference": "first-half",
"months": ["2026-01", "2026-06"],
"rate": "0.08",
"cap": "4000.00",
"deduction": "400.00",
"free": "1200.00",
"upper": "3000.00",
"middle_rate": "0.14",
"upper_rate": "0.18"
}
Assistant
calculate(program="employee_contribution = round(min(1430.25, 4000.00) * 0.08, 2)\ntaxable_pay = 1430.25 - employee_contribution - 115.82\ntax_free = 1200.00\nupper_boundary = 3000.00\nmiddle_rate = 0.14\nupper_rate = 0.18\nbracket1 = max(0, min(taxable_pay, tax_free))\nbracket2 = max(0, min(taxable_pay, upper_boundary) - tax_free)\nbracket3 = max(0, taxable_pay - upper_boundary)\nincome_tax = round(bracket2 * middle_rate + bracket3 * upper_rate, 2)\nnet_pay = 1430.25 - employee_contribution - 115.82 - income_tax")Tool
{
"results": {"employee_contribution": "114.42", "taxable_pay": "1200.01", "tax_free": "1200.00", "upper_boundary": "3000.00", "middle_rate": "0.14", "upper_rate": "0.18", "bracket1": "1200.00", "bracket2": "0.01", "bracket3": "0", "income_tax": "0.00", "net_pay": "1200.01"}
}
Assistant
{
"decision": "calculate",
"currency": "LCR",
"gross_pay": "1430.25",
"employee_contribution": "114.42",
"allowed_deduction": "115.82",
"taxable_pay": "1200.01",
"income_tax": "0.00",
"net_pay": "1200.01",
"clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]
}