We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results). We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs. Our other findings: - Luna performs better than Jev out of the box, but the gap disappears on harder cases. - Jev's confidence is better calibrated. Most of its mistakes come with low confidence its right. Most of Luna's mistakes come with h