Luma Payroll Gym
Rules Report
← Training / β 0.001, rubric 7
β 0.001, rubric 7
β 0.001, rubric 5 β 0.04, rubric 5 β 0.04, rubric 3 β 0.04, rubric 4 β 0.04, rubric 6 β 0.001, rubric 7 β 0.04, rubric 7 β 0.04, rubric 7, answer-weighted SFT + RL, β 0.001 SFT + RL, β 0.001, seed 8 SFT + RL, β 0.001, seed 9 SFT + RL, β 0.04 SFT + RL, β 0.04, balanced SFT + RL, β 0.04, more signal SFT + RL, β 10 SFT + RL, β 100
KL β
0.001
Rubric
7
Steps
60 / 60
LR
1e-05
Group size
4-16
Epochs
2
Prompt
decision examples, thinking disabled
GPU time
4.9h
Training curves, 60 steps

Mean reward

-50% 0% 50% Step 0 Step 30 Step 60

KL divergence (k3)

0 0.05 0.1 Step 0 Step 30 Step 60

Policy entropy

0 0.2 0.4 Step 0 Step 30 Step 60

Completion length (tokens)

0 50 100 150 Step 0 Step 30 Step 60

Trap pass rate

0% 50% 100% Step 0 Step 30 Step 60

Payslip pass rate

0.0% 2.0% 4.0% 6.0% Step 0 Step 30 Step 60

Correct clarifications

0% 50% 100% Step 0 Step 30 Step 60

Calculates through traps

0% 50% 100% Step 0 Step 30 Step 60

Non-tied groups

0% 50% 100% Step 0 Step 30 Step 60
Sampled completions
Step 1
generated-1001-00064 · Calculation
-50% retrieve · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"], "explanation": "The task requires retrieving information from the rulebook based on the provided month, and R1 and R2 are needed for the calculation. These are the required tools and sources for the task."}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6717.67", "employee_contribution": "1075.95", "allowed_deduction": "326.05", "taxable_pay": "1075.45", "income_tax": "65.25", "net_pay": "1061.30", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6717.67", "employee_contribution": "1103.67", "allowed_deduction": "326.05", "taxable_pay": "1824.98", "income_tax": "39.30", "net_pay": "1794.38", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6717.67", "employee_contribution": "1001.63", "allowed_deduction": "1001.63", "taxable_pay": "1822.25", "income_tax": "42.25", "net_pay": "4794.49", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7", "R9"]}
generated-1001-00055 · Calculation
-50% retrieve · wrong decision
{"decision": "retrieve", "field": "2026-11", "value": "2891.48", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "2891.48", "employee_contribution": "144.48", "allowed_deduction": "0.00", "taxable_pay": "2314.00", "income_tax": "38.50", "net_pay": "2063.98", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{ "decision": "calculate", "currency": "LCR", "gross_pay": "2891.48", "employee_contribution": "1727.00", "allowed_deduction": "0.00", "taxable_pay": "2058.48", "income_tax": "107.00", "net_pay": "2174.48", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7", "R9"] }
-50% retrieve · wrong decision
{"decision": "retrieve", "field": "2026-11-05", "value": "2891.48", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7"]}
generated-1001-00092 · Trap CONFLICTING_GROSS_PAY
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "6265.40", "employee_contribution": "217.00", "allowed_deduction": "217.79", "taxable_pay": "2178.79", "income_tax": "43.40", "net_pay": "6040.10", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% clarify · wrong clarification
{"decision": "clarify", "reason_code": "MISSING_GROSS_PAY", "clauses": ["R1", "R7"], "explanation": "Gross pay is missing. Please provide the monthly gross pay before calculation."}
0% clarify · wrong clarification
{"decision": "clarify", "reason_code": "MISSING_GROSS_PAY", "clauses": ["R1", "R7"], "explanation": "Gross pay is missing. Please provide the monthly gross pay before calculation."}
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "6265.40", "employee_contribution": "1308.26", "allowed_deduction": "217.79", "taxable_pay": "2100.51", "income_tax": "210.21", "net_pay": "4062.20", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7"]}
generated-1001-00096 · Trap MISSING_GROSS_PAY
-50% retrieve · invalid format
{"decision": "retrieve", "field": "contribution_rate", "value": "0.08", "clauses": ["R1", "R3", "R4"], "clauses": ["R1", "R3", "R4", "R5"]}
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "2400.00", "employee_contribution": "120.00", "allowed_deduction": "100.00", "taxable_pay": "2180.00", "income_tax": "118.00", "net_pay": "2062.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6", "R7"]}
-50% calculate · wrong decision
{ "decision": "calculate", "currency": "LCR", "gross_pay": "2400.00", "employee_contribution": "120.00", "allowed_deduction": "100.00", "taxable_pay": "2180.00", "income_tax": "118.00", "net_pay": "2062.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"] }
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "1800.00", "employee_contribution": "102.33", "allowed_deduction": "102.33", "taxable_pay": "2142.00", "income_tax": "429.33", "net_pay": "1789.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
Step 30
generated-1001-00028 · Calculation
-50% clarify · wrong decision
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The provided payroll month does not match the reference month, so no reference row can be selected. Please ensure the correct month is used for the reference."}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6216.27", "employee_contribution": "1001.90", "allowed_deduction": "1001.90", "taxable_pay": "1117.68", "income_tax": "45.28", "net_pay": "5210.99", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6216.27", "employee_contribution": "206.26", "allowed_deduction": "206.66", "taxable_pay": "2060.00", "income_tax": "205.66", "net_pay": "2070.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6216.27", "employee_contribution": "1040.00", "allowed_deduction": "206.66", "taxable_pay": "2266.00", "income_tax": "406.00", "net_pay": "4162.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
generated-1001-00088 · Calculation
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "8340.45", "employee_contribution": "2245.10", "allowed_deduction": "284.60", "taxable_pay": "2213.50", "income_tax": "1085.15", "net_pay": "6120.85", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "8340.45", "employee_contribution": "2259.45", "allowed_deduction": "284.60", "taxable_pay": "2246.45", "income_tax": "68.45", "net_pay": "6090.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "8340.45", "employee_contribution": "2250.45", "allowed_deduction": "284.60", "taxable_pay": "2375.45", "income_tax": "84.45", "net_pay": "5965.50", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
-50% clarify · wrong decision
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing."}
generated-1001-00005 · Trap MISSING_PAYROLL_MONTH
93% clarify · scored
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The provided month (2027-02) does not match any reference row for 2026. Please confirm the correct reference row."}
93% clarify · scored
{ "decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The specified payroll month (2027-02) does not have a reference row. No reference row can be used for this calculation." }
100% clarify · scored
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing, so no reference row can be chosen. Please provide an authoritative schedule that covers it."}
100% clarify · scored
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing, so no reference row can be chosen. Please provide the payroll month."}
generated-1001-00002 · Trap NO_APPLICABLE_REFERENCE
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "103.59", "allowed_deduction": "66.18", "taxable_pay": "1261.62", "income_tax": "55.62", "net_pay": "1104.05", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
100% clarify · scored
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "No reference row covers this payroll month. Please provide an authoritative schedule that covers it."}
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "70.89", "allowed_deduction": "66.18", "taxable_pay": "1728.12", "income_tax": "43.38", "net_pay": "1102.31", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
-50% calculate · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "77.59", "allowed_deduction": "66.18", "taxable_pay": "1258.01", "income_tax": "83.73", "net_pay": "1004.50", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
Step 60
generated-1001-00085 · Calculation
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9993.09", "employee_contribution": "127.09", "allowed_deduction": "127.12", "taxable_pay": "2188.00", "income_tax": "59.00", "net_pay": "9805.99", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9993.09", "employee_contribution": "127.09", "allowed_deduction": "127.12", "taxable_pay": "1250.00", "income_tax": "125.00", "net_pay": "1966.01", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9993.09", "employee_contribution": "127.09", "allowed_deduction": "127.12", "taxable_pay": "1296.17", "income_tax": "43.25", "net_pay": "1406.88", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9993.09", "employee_contribution": "1208.13", "allowed_deduction": "127.12", "taxable_pay": "2170.95", "income_tax": "507.95", "net_pay": "3815.95", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
generated-1001-00043 · Calculation
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "4681.59", "employee_contribution": "151.01", "allowed_deduction": "168.38", "taxable_pay": "1688.12", "income_tax": "42.01", "net_pay": "1304.58", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "4681.59", "employee_contribution": "168.17", "allowed_deduction": "168.38", "taxable_pay": "2367.12", "income_tax": "1133.00", "net_pay": "3043.42", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "4681.59", "employee_contribution": "1258.03", "allowed_deduction": "168.38", "taxable_pay": "2370.03", "income_tax": "475.03", "net_pay": "2403.56", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% calculate · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "4681.59", "employee_contribution": "1679.23", "allowed_deduction": "168.38", "taxable_pay": "2161.90", "income_tax": "42.00", "net_pay": "3040.60", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
generated-1001-00101 · Trap MISSING_PAYROLL_MONTH
0% clarify · wrong clarification
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "The payroll month is not covered by any reference table. Please provide an authoritative schedule that covers 2027-02."}
100% clarify · scored
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing, so no reference row can be chosen. Please provide an authoritative schedule that covers it."}
100% clarify · scored
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing, so no reference row can be chosen. Please provide an authoritative schedule that covers it."}
-50% invalid · invalid format
{"decision": "clarify", "reason_code": "MISSING_PAYROLL_MONTH", "clauses": ["R1", "R7"], "explanation": "The payroll month is missing. Please provide the payroll month.0" } We are given the following information: - Gross pay: 8215.02 - Contract gross pay: 8215.02 (same as gross pay) - Reference month: 2026-02 (but note, there is a discrepancy with the contract gross pay) Since the contract gross pay and the actual gross pay are identical, there is no conflict. However, there is a reference table entry for 2026-02, which might not cover the entire year. The primary reference for 2026-02 is not fully defined. Also, there is no reference for 2027-02, which means no reference row can be used for 2027-02. The payroll month specified in the task is 2026-02, but the reference table only has 2026-02 and 2026-01-03. The task requires a reference row to calculate the pay, which is not available for 2027-02. The task
generated-1001-00041 · Trap NO_APPLICABLE_REFERENCE
100% clarify · scored
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "No reference row covers this payroll month. Please provide an authoritative schedule that covers it."}
100% clarify · scored
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "No reference row covers this payroll month. Please provide an authoritative schedule that covers it."}
100% clarify · scored
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "No reference row covers this payroll month. Please provide an authoritative schedule that covers it."}
100% clarify · scored
{"decision": "clarify", "reason_code": "NO_APPLICABLE_REFERENCE", "clauses": ["R1", "R7"], "explanation": "No reference row covers this payroll month. Please provide an authoritative schedule that covers it."}