Luma Payroll Gym
Rules Report
← Training / β 0.04, rubric 4
β 0.04, rubric 4
β 0.001, rubric 5 β 0.04, rubric 5 β 0.04, rubric 3 β 0.04, rubric 4 β 0.04, rubric 6 β 0.001, rubric 7 β 0.04, rubric 7 β 0.04, rubric 7, answer-weighted SFT + RL, β 0.001 SFT + RL, β 0.001, seed 8 SFT + RL, β 0.001, seed 9 SFT + RL, β 0.04 SFT + RL, β 0.04, balanced SFT + RL, β 0.04, more signal SFT + RL, β 10 SFT + RL, β 100
KL β
0.04
Rubric
4
Steps
14 / 100
LR
2e-05
Group size
4-4
Epochs
1
Prompt
balanced three-example scaffold, thinking disabled
GPU time
0.2h
Training curves, 14 steps

Mean reward

0.0% 2.0% 4.0% 6.0% Step 0 Step 50 Step 100

KL divergence (k3)

0 0.002 0.004 0.006 Step 0 Step 50 Step 100

Policy entropy

0 0.1 0.2 0.3 Step 0 Step 50 Step 100

Completion length (tokens)

0 50 100 Step 0 Step 50 Step 100

Trap pass rate

0.0% 2.0% 4.0% 6.0% Step 0 Step 50 Step 100

Payslip pass rate

0.0% 2.0% 4.0% 6.0% Step 0 Step 50 Step 100

Non-tied groups

0% 20% 40% 60% Step 0 Step 50 Step 100
Sampled completions
Step 1
generated-1001-00075 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "employee_contribution", "value": "0.05", "clauses": ["R1", "R3", "R4"]}
5% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "976.76", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "950.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
5% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "976.76", "employee_contribution": "50.00", "allowed_deduction": "350.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "976.76", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · invalid format
{"decision": "calculate", "currency": "LCR", "gross_pay": "976.76", "employee_contribution": "50.00", "allowed_deduction": "350.00", "taxable_pay": "950.00", "net_pay": "950.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
generated-1001-00070 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.05", "clauses": ["R1", "R3"]}
0% invalid · invalid format
{"decision": "clarify", "reason_code": "MISSING_GROSS_PAY", "explanation": "Gross pay is missing. Please provide the monthly gross pay before calculation."}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "7022.36", "employee_contribution": "50.00", "allowed_deduction": "31.67", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "950.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.05", "clauses": ["R1", "R3"]}
generated-1001-00071 · Trap
0% invalid · wrong decision
{"decision": "retrieve", "field": "allowed_deduction", "value": "400.00", "clauses": ["R1", "R3", "R4"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "3500.00", "clauses": ["R1", "R3", "R4", "R5", "R6"]}
0% invalid · invalid format
{"decision": "calculate", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"], "gross_pay": "3713.85", "employee_contribution": "164.48", "allowed_deduction": "400.00", "taxable_pay": "1000.00", "income_tax": "0.00", "net_pay": "3713.85", "reason": "Mention the correct contribution rate for the payroll month for accurate calculation."}
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "3500.00", "clauses": ["R1", "R3", "R4"]}
generated-1001-00054 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.03", "clauses": ["R1", "R3"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "digit", "value": "0.04", "clauses": ["R1", "R5", "R6", "R7"]}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6750.61", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "7251.50", "income_tax": "0.00", "net_pay": "7251.50", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
Step 7
generated-1001-00076 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1", "R3"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "4000.00", "clauses": ["R1", "R3", "R4"]}
5% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.23", "employee_contribution": "45.75", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "1175.50", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · invalid format
{"decision": "retrieve", "field": "contribution_cap", "value": "350.00", "clauses": ["R1", "R4"], "reason": "Clarity of R1 and R4 indicates contribution rates and caps for the selected half."}
generated-1001-00040 · Calculation
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6173.69", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "5703.69", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6173.69", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "6173.69", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1", "R3"]}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "6173.69", "employee_contribution": "0.04", "allowed_deduction": "0.00", "taxable_pay": "0.00", "income_tax": "0.00", "net_pay": "6173.69", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
generated-1001-00104 · Trap
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R3", "R4"]}
0% invalid · invalid format
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R4", "R6"], "clauses": ["R1", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{ "decision": "retrieve", "field": "contribution_rate", "value": "0.05", "clauses": ["R1", "R3"] }
0% invalid · wrong decision
{ "decision": "calculate", "currency": "LCR", "gross_pay": "8596.51", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "950.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"] }
generated-1001-00019 · Calculation
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9221.00", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "9750.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9221.00", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "9221.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{ "decision": "retrieve", "field": "contribution_rate", "value": "0.05", "clauses": ["R1", "R3"] }
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9221.00", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "9750.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
Step 14
generated-1001-00097 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1", "R3"]}
0% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "9000.07", "employee_contribution": "0.05", "allowed_deduction": "0.00", "taxable_pay": "9075.00", "income_tax": "0.00", "net_pay": "9075.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{ "decision": "retrieve", "field": "employee_contribution_rate", "value": "0.04", "clauses": ["R1", "R3", "R4"] }
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.04", "clauses": ["R1", "R3"]}
generated-1001-00004 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R2", "R3", "R4", "R6"]}
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R3", "R4"]}
5% invalid · scored
{"decision": "calculate", "currency": "LCR", "gross_pay": "936.04", "employee_contribution": "108.88", "allowed_deduction": "108.88", "taxable_pay": "0.00", "income_tax": "0.00", "net_pay": "936.04", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{ "decision": "retrieve", "field": "employee_contribution", "value": "108.00", "clauses": ["R1", "R3", "R4"] }
generated-1001-00033 · Calculation
0% invalid · wrong decision
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R4", "R7"]}
0% invalid · wrong decision
{ "decision": "retrieve", "field": "contribution_rate", "value": "0.08", "clauses": ["R1", "R3"] }
0% invalid · wrong decision
{"decision": "retrieve", "field": "contribution_rate", "value": "0.06", "clauses": ["R1", "R3"]}
5% invalid · scored
{ "decision": "calculate", "currency": "LCR", "gross_pay": "1052.65", "employee_contribution": "50.00", "allowed_deduction": "190.09", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "950.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"] }
generated-1001-00002 · Trap
0% invalid · invalid format
{"decision": "retrieve", "field": "deduction_cap", "value": "350.00", "clauses": ["R1", "R4", "R7", "R9"], "explanation": "This is the deduction cap for the first half. The deduction cap is 350.00. The reference table shows the first half's deduction cap as 350.00, which matches this result. The second half's deduction cap is 350.00, which is also the same as the deduction cap for the first half. The reference table shows the deduction cap for the second half as 350.00, which matches the result. The deduction cap for the first half is 350.00, and the deduction cap for the second half is 350.00, which is the same. The reference table shows the deduction cap for the first half as 225.00 and for the second half as 350.00, but the provided values are 350.00. The reference table shows the deduction cap for the first half as 200.00 and for the second half as 350.00, but the provided values are 350.00. The reference table shows the deduction cap for the first half as
0% invalid · wrong decision
{ "decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "50.00", "allowed_deduction": "66.18", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "1170.19", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"] }
0% invalid · invalid format
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "50.00", "allowed_deduction": "0.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "1170.19 - 50.00 + 0.00 = 1170.19", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}
0% invalid · wrong decision
{"decision": "calculate", "currency": "LCR", "gross_pay": "1170.19", "employee_contribution": "50.00", "allowed_deduction": "50.00", "taxable_pay": "950.00", "income_tax": "0.00", "net_pay": "1070.00", "clauses": ["R1", "R2", "R3", "R4", "R5", "R6"]}