24x faster, 9x cheaper, and I'm keeping my old LLM judge

24x faster. 9x cheaper. But I'm keeping my old LLM judge for now.

Earlier this week I replayed every decision one of my agentic systems ever made through Jev.

The system under test is a multi-step agent whose output is long-form, verbose content. It uses an LLM-as-a-judge gate to check responses for rule violations (fabricated claims, unsourced numbers, injection artifacts, tone breaks, etc.). Median latency is ~6 seconds and ~$0.003 per decision.

Then I rebuilt the exact judge inputs from the pipeline's checkpoints and ran each one through both models on identical content. Jev answers in ~250 ms, at $0.00034 per decision.

Interestingly, when judging verbose text against a complex rulebook, Jev hedged. A huge chunk of its "pass" answers landed in the 0.29 to 0.64 probability window, pure coin-flip territory. It agreed with the LLM judge 55% of the time, and every single disagreement happened near p ~ 0.5.

What I learned is that wording is load-bearing. So, for now, this particular gate keeps its LLM judge.

Classification-shaped decisions (tool gating, routing, retry triage) are exactly where Jev shines with decisive probabilities in milliseconds and micro cents. Evaluating long, nuanced content against complex guidelines may still belong to a heavier LLM. And the gate above both stays human.

Next I want to try Jev for reranking in RAG pipelines. I am excited about it and will share findings soon.