Awesome JEV

Papers, open models and evaluations behind System One models, the class of AI that answers typed questions with calibrated probabilities in one pass.

90 entries
Jev61The model, its open-source rebuilds, the independent evaluations, and the commentary.

System One & Jev 19

What TypeSafe has published: the launch post, the docs, the SDKs, the essays, and the failure modes it admits to.

Jev 1.13 jaggedness

official

TypeSafe's documented failure modes: literal reading, counting, dates, indirection, distractor state, adversarial content

System One Adapter

2026

Official drop-in that serves the same typed interface from OpenAI or Anthropic models, the baseline for every comparison

72

1,850 points and 485 comments; the CEO confirms the zero-shot classifier reading and the encoder-with-heads shape

System One (concept)

official

Defines the model class: fast structured decisions for software, contrasted with text-generating LLMs

Confidence

official

Confidence as a statistic of the returned distribution, and the act, confirm or escalate thresholds built on it

Patterns

official

Speculative fan-out, confidence-gated routing, composite scoring and intent routing as architectural patterns

Cookbooks

official

Reproducible recipes for parallel questions, reranking, guardrails, extraction and hierarchical classification

The first third-party surface: Jev-latest as an evaluation model behind experimental_evaluate, no waitlist

Manifesto

official

Machine-native composable AI as reliable primitives for programmers, against the assistant framing

The Bitterest Lesson

official

TypeSafe's own essay ranking task choice above data, compute and algorithms, the argument behind System One Models

Open Source 18

Open weights and code that rebuild the System One shape from encoders, small decoders and constrained decoding.

Can something like Jev run on a 3090 at home; open baseline for direct typed option scoring

From-scratch model with Jev's exact shape: text plus N options in, one probability per option out, option-attention head, Doom and chess demos

608

Qwen-2.5-1B-RLCD

2026model

Qwen2.5-1.5B fine-tune plus parallel constrained decoding; all schema fields scored in one broadcast prefill, 5.6x to 7x faster on Apple Silicon

Decorator that routes DSPy typed Signatures to Jev where the signature is a pure decision

56

Jev-compatible API endpoint served from open models with prefill-only inference

16

Unofficial study of Jev-style parallel typed decisions on stock 1.5B to 8B models on Apple Silicon, with benchmarks

One-pass scorer over a variable option list: hashed n-gram encoder, rival-aware attention, gated head, temperature scaling

Qwen3.5-2B fine-tune that emits typed decisions with calibrated probabilities in one pass

Local bilingual probability decisions from context, questions and candidate answers

LitJev

2026

Jev's decision layer on off-the-shelf Qwen checkpoints: one shared prefill, then option logits read per question, no training

Choice, Score and Noul on Qwen3.5-0.8B, the smallest decoder-based reproduction

LFM2.5-2.6B-RLCD

2026model

RLCD-style fine-tune of Liquid AI's LFM2.5-2.6B, the largest open attempt so far

system-one-mini

2026model

DistilBERT-sized System One shape, a floor for how small the idea can go

Independent Evaluations 17

Every independent test of Jev published so far, with the headline number where the source gives one. TypeSafe's own dashboard is listed and marked official.

Jev as reranker over 8 datasets and 1,617 questions: nDCG@10 0.692 versus Cohere Rerank 4 Pro 0.691, at 422ms

Blind prompt-injection run on 662 deepset messages: 96.5% accuracy, 0.9927 ROC-AUC, ECE 0.0588, p50 325ms

Jev routes 100 mocked tools behind a confidence gate, measuring steps, tool calls, tokens and cost against the LLM choosing directly

45

Claim-level Brier scoring of federal motion-to-dismiss outcomes, a fixed binary with one probability per unit; no Jev row published yet

Three production SaaS decisions published as schemas with auto-post confidence thresholds; LLM baseline rows filled, Jev row still empty

Zero-shot spam Noul on 18,514 emails reaching 0.9833 accuracy, matching a TF-IDF classifier trained on 14,800 labels

100 balanced secret-detection cases as single Noul questions scored by accuracy, AUC and Brier; server p50 75 to 90ms

Reproducible harness scoring Jev ultrafast research-browser runs over 11 baseline cases plus 18 human and quant stress cases with QC grades

Tic-tac-toe and connect four pitting Jev against four frontier models on identical legal-move choice options; no aggregate results published yet

777 judgments over 37 articles in 0.7s for a quarter of a cent; caught six of seven planted defects, Fable seven

One direct Jev question per row against 12 to 14 Jev-scored dimensions with fitted weights on three tasks: 0.9076 vs 0.8373 on Japanese NLI, but 25x the hard-benign false positives, 37.2% vs 1.5%

Commentary & Analysis 7

Reporting and technical commentary that checks the launch claims against the evidence.

Claim-by-claim audit separating verified Jev pricing and latency from unproven calibration; puts aggregate accuracy at 67.8% versus Opus 5's 73.1%

Secondary analysis of TypeSafe's dashboard putting Jev at about 67.8% mean agreement against 74.1% for the best comparator

Lineage29Twenty-nine earlier works, each labelled with why it is here: named by TypeSafe or the launch discussion, or the same shape as Jev.

The Shape Before Jev 15

Earlier work with the same input and output shape: a fixed answer set, one probability per option, no generated text. Label-conditioned encoders, scalar reward heads, reinforcement learning for calibrated confidence, and the single-pass inference TypeSafe's own forks point at.

Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach

same shape Label set given at inference, an entailment model returns one probability per label, no text generated

Training language models to follow instructions with human feedback

same shape A Bradley-Terry head emits one scalar per response in a single pass, no text, co-authored by Jev's founder

RLCR

ICLR 20262025

Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty

same shape Adds a Brier-score reward to RLVR so the model emits calibrated confidence, the closest published relative of TypeSafe's RLCD

LLaDA

2025

Large Language Diffusion Models

primary The typesafe-ai GitHub org forked this masked diffusion LM, the strongest public hint at how Jev fills every answer slot in one pass

GLiNER

2023

Generalist Model for Named Entity Recognition using Bidirectional Transformer

named The span-and-label encoder family HN mapped Jev onto, types supplied at inference and scored in one bidirectional pass

GLiClass

2025

Generalist Lightweight Model for Sequence Classification Tasks

named The open analogue HN pointed at: labels and text in one encoder pass, one probability per label, no decoding

Passage Re-ranking with BERT

same shape Landmark cross-encoder: pair in, one scalar relevance probability out, no generation, the ancestor of Jev's Score primitive

Generative or Discriminative?

EMNLP 20252025

Revisiting Text Classification in the Era of Transformers

same shape Controlled comparison of encoder, autoregressive and diffusion classifiers over fixed label sets on accuracy, calibration and ordinality

LLM-based Input-Output Safeguard for Human-AI Conversations

same shape Fixed safety taxonomy with the verdict read off one safe/unsafe token probability, the guardrail classifier Noul replaces

Rewarding Doubt

ICLR 20262025

A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models

same shape Trains confidence expression by RL on the logarithmic scoring rule, an independent rediscovery of the proper-scoring-rule reward RLCD uses

Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning

same shape RL that adjusts decision-token probabilities directly, keeping RLVR accuracy while cutting ECE, the closest public analogue of Jev's typed decision heads

vLLM

SOSP 20232023

Efficient Memory Management for Large Language Model Serving with PagedAttention

primary The typesafe-ai GitHub org forked this engine; paged KV cache plus prefix caching is what makes extra questions over one shared state nearly free

Mercury

2025

Ultra-Fast Language Models Based on Diffusion

named HN read Jev as a stripped down text diffusion model, and Mercury is that idea shipped commercially with parallel refinement

What Jev Is Sold Against 9

The tools TypeSafe and the launch discussion named as what Jev replaces: constrained decoding, structured outputs, typed prompt programming, LLM judges, routers and guard classifiers.

Efficient Guided Generation for Large Language Models

named The finite-state-machine guided decoding HN named as the incumbent way to get typed values, which Jev claims to replace

DSPy

ICLR 20242023

Compiling Declarative Language Model Calls into Self-Improving Pipelines

named Typed signatures compiled into prompts, named on the HN thread as the fair comparison for Jev's typed question interface

Learning to Route LLMs with Preference Data

same shape Router scores a fixed two model set and returns win probability per option before any text is generated

named Constrained generation library named on the HN launch thread as what Jev's typed outputs get compared against

21.8k

named The provider-side JSON-schema guarantee the CEO named on HN as what Jev replaces, shape enforced but no probability returned

Let Me Speak Freely?

EMNLP 2024 Industry Track2024

A Study on the Impact of Format Restrictions on Performance of Large Language Models

named Measures the accuracy format restrictions cost, the study behind the CEO's HN claim that constrained decoding makes models dumber

A Rigorous Benchmark of Structured Outputs for Language Models

primary 10k real schemas scored on validity, coverage and latency, the constrained-decoding route Jev's 0% type errors claim competes against

MT-Bench

NeurIPS 20232023

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

named The landmark LLM-as-a-judge paper, named on the launch thread as the layer Jev's score and Noul primitives replace

Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming

named, same shape Input and output classifiers gating a frontier model on a fixed policy, the deployed slot Noul targets

Where the Name Comes From 5

System 1 in Kahneman's sense, the bitter lesson TypeSafe argues with, and the Jevons paradox the model is named after.

named The source TypeSafe cites for naming Jev after System 1, fast intuitive judgement with no deliberation

named The essay TypeSafe's own Bitterest Lesson argues against, named in TypeSafe's materials as its starting point