I ran 104 of my emails through Jev. I'm still not switching
Jev, the new model from TypeSafe AI, answers typed questions with calibrated probabilities instead of text. On 104 of my real emails it was four times faster than Gemini Flash-Lite and nearly as accurate, and as a pre-filter it would have skipped 35–42% of LLM calls. Why my email agent still doesn't use it.
I ran 104 of my real emails through Jev, the new model from TypeSafe AI. A quarter of a second per email, a fraction of a cent. I’m still not switching. Here’s why.
The one decision that matters
My open-source email agent decides which mail reaches my Telegram right away and which waits for the evening digest. That one decision is the whole product: a wrong “ignore” is a missed bill, a wrong “important” is noise.
A model that doesn’t write
Jev is unusual: it doesn’t write text. You ask typed questions (“which of these four levels?”, “does this rule apply?”) and get calibrated probabilities back.
The test
I added it as a fourth provider (the integration) and ran the same emails through Claude Sonnet as the reference, Gemini Flash-Lite (what I run today) and Jev. The sample: the 104 most recent emails from four personal inboxes that reached the model. “Notify” means important or above; all three got the same inputs and the same ignore rules.
- Agreed with Sonnet on “notify or not”: Gemini 91%, Jev 87%.
- Median time per email: Sonnet 3.2 s, Gemini 0.96 s, Jev 0.24 s.
- Languages: Russian and Polish mail works. Jev trailed Gemini on Polish, but on English too, so the gap isn’t about language.
- The surprise: as a pre-filter it shines. Skipping the LLM whenever Jev was confident a mail was noise would have saved 35–42% of calls without losing a single important email: 35% when Jev is at least 90% sure, 42% when it’s at least 50% sure.
Why I’m not switching
Besides the accuracy gap, the data terms, as of 2 October 2026:
- Hosting is US-only, with no announced EU region (privacy policy, sections Retention and International Visitors; subprocessors).
- Zero data retention is offered to enterprise customers only (TypeSafe docs).
- The customer agreement (§4.1 and §4.3) grants a perpetual license to derive telemetry from customer data, though not to train on it.
That’s why my work inbox stayed out of the test.
For fairness: Gemini’s paid API keeps prompts for a limited time, only for abuse monitoring and legal requests, and doesn’t use them to improve products, but it doesn’t promise EU-only storage either (Gemini API terms).
Small sample, early-access model. An EU region or zero data retention for smaller customers would change my mind.
Have you tried decision models like Jev? Where do they fit in your stack?
