Luna 2 | See The Total Cost of Your Evals At Scale

Inputs

Agent archetype

• SimpleSingle-turn RAG · 3K tok
• Tool-Using ReAct + tools · 10K tok
• Code / Research Multi-step · 50K tok

Evaluator model

• $3.25/1M

• GPT-5 Minipopular: $0.25 / $2.00−5% F1 accuracy

• GPT-4o-minipopular: $0.15 / $0.60−5% F1 accuracy

• Gemini 3 Flash: $0.50 / $3.00

• GPT-4o: $2.50 / $10.00

More models

• Claude Sonnet 4.6: $3.00 / $15.00
• Claude Opus 4.7: $5.00 / $25.00
• GPT-5.5: $5.00 / $30.00

Custom Edit rates

• Effective rate at 90% input / 10% output: $3.25/1M

Number of agents

• 5
  • 150

Traces per agent / day

• 40,000

Baseline shared metrics

• 5
  • 020

Custom metrics per agent

• 3
  • 010

Metric scope

• Pooled
• Per-agent

Annual savings with Luna

• $45.73M

LLM-as-judge costs 27× more than Luna at this scale

Per-eval response latency

• Luna 2: 80ms
• GPT-4o: 2.5s
• Inline guardrail viable on Luna · GPT-4o is offline-only
• 31× faster

Luna is faster per evaluation

Tokens scanned / mo

• 1.22T

Trace volume

• 6M traces/mo
• 5 agents · 40,000 traces/day each

LLM-as-judge cost

• $3.96M/mo
• $47.48M annual

Luna cost

• $146.1K/mo
• $1.75M annual

Same budget. Different coverage.

• Luna 2: 100%
• GPT-4o: 3.69%

If you pin the budget at Luna's cost ($146.1K/mo)

• 96%

Blind spot with judge at equal budget

Annual eval cost vs. # agents

Methodology & sources