JevCheeseDecision models, in context.

Public accuracy: 86.1% · Sealed accuracy: 37.0% · Gap: 49.1 pp

Licence: Apache-2.0 (author adapter and server metadata)

Public accuracy: 89.6% · Sealed accuracy: 38.0% · Gap: 51.6 pp

Licence: Apache-2.0 (weights and code; NOTICE credits JevK5, Qwen and SemIf); base Qwen3.5-4B Apache-2.0

Public accuracy: 83.5% · Sealed accuracy: 34.7% · Gap: 48.8 pp

Licence: Apache-2.0 (package and weights)

Public accuracy: 86.6% · Sealed accuracy: 36.7% · Gap: 49.9 pp

Underlying model: closed

Licence: proprietary API

Public accuracy: 85.3% · Sealed accuracy: 33.1% · Gap: 52.2 pp

Licence: Apache-2.0 (code/adapter); Apache-2.0 (Qwen3.5-4B base)

Public accuracy: 87.9% · Sealed accuracy: 33.8% · Gap: 54.1 pp

Licence: shim MIT; weights Apache-2.0 with Google's Gemma Prohibited Use Policy

Public accuracy: 82.3% · Sealed accuracy: 34.1% · Gap: 48.2 pp

Underlying model: Qwen3.5-4B plus HopitAI/hopper LoRA

Licence: Component-specific terms recorded in RESULT.md; submitted adapter release and Qwen base retain their respective terms

Public accuracy: 85.7% · Sealed accuracy: 33.1% · Gap: 52.6 pp

Underlying model: google/gemma-4-12B-it LoRA fine-tune, merged and exported as Q8_0 GGUF

Licence: Apache-2.0, including the applicable Gemma 4 base/derivative licence terms

Public accuracy: 79.2% · Sealed accuracy: 28.2% · Gap: 51.0 pp

Underlying model: Qwen/Qwen3.5-4B + kshetrajna12/reflex-qwen3.5-4b-lora

Licence: MIT (code, adapter); Apache-2.0 (base)

Public accuracy: 84.0% · Sealed accuracy: 29.9% · Gap: 54.1 pp

Underlying model: inference method on google/diffusiongemma-26B-A4B-it (one structured denoising read), not a separately trained model

Licence: Apache-2.0 code; Google DiffusionGemma Apache-2.0 weights; no djev-specific weights

Public accuracy: 88.7% · Sealed accuracy: 32.1% · Gap: 56.6 pp

Underlying model: google/gemma-4-12B-it fine-tuned and merged, with a trained 256-way decision head

Licence: Apache-2.0, following Gemma 4; dataset rights stated separately by the author

Public accuracy: 79.7% · Sealed accuracy: 27.6% · Gap: 52.1 pp

Underlying model: Qwen3.5-4B merged r16 LoRA, candidate-logit readout

Licence: Apache-2.0

Public accuracy: 81.0% · Sealed accuracy: 26.3% · Gap: 54.7 pp

Underlying model: Qwen/Qwen3.5-4B (frozen, BF16)

Licence: MIT (code); Qwen3.5 weights Apache-2.0

Public accuracy: 81.0% · Sealed accuracy: 25.6% · Gap: 55.3 pp

Underlying model: Qwen/Qwen3.5-4B frozen BF16 backbone

Licence: MIT code; Apache-2.0 weights

Public accuracy: 80.5% · Sealed accuracy: 26.0% · Gap: 54.5 pp

Underlying model: Qwen/Qwen3.5-4B text model, zero-shot next-token letter probabilities

Licence: MIT code; Apache-2.0 Qwen weights

Public accuracy: 73.2% · Sealed accuracy: 27.6% · Gap: 45.6 pp

Underlying model: google/gemma-4-E2B-it + LoRA

Licence: MIT (repository LICENSE; Gemma weights keep Google’s terms)

Public accuracy: 79.2% · Sealed accuracy: 26.6% · Gap: 52.6 pp

Underlying model: Qwen/Qwen3.5-4B with a LoRA, read at the option-letter logits with a fitted temperature (no extra head)

Licence: Apache-2.0 (code and weights); base Qwen/Qwen3.5-4B Apache-2.0

Public accuracy: 74.9% · Sealed accuracy: 23.4% · Gap: 51.5 pp

Licence: CC-BY-NC-4.0, research use only (XNLI and RACE in the training mix)

Public accuracy: 80.1% · Sealed accuracy: 28.2% · Gap: 51.8 pp

Underlying model: Qwen/Qwen3-32B, bf16, read as a direct-logit classifier (no fine-tuning)

Licence: Apache-2.0 (Qwen3-32B weights); serving code public

Public accuracy: 68.0% · Sealed accuracy: 29.9% · Gap: 38.1 pp

Underlying model: 4B instruction-aware generative reranker

Licence: Apache-2.0

Public accuracy: 83.1% · Sealed accuracy: 31.5% · Gap: 51.6 pp

Underlying model: Qwen3.5-35B-A3B-Base with a trained decision readout, 34.7B parameters / 3B active

Licence: Apache-2.0

Public accuracy: 69.7% · Sealed accuracy: 27.3% · Gap: 42.4 pp

Underlying model: Qwen3-4B-Instruct-2507 BF16

Licence: Apache-2.0

Public accuracy: 78.4% · Sealed accuracy: 26.3% · Gap: 52.1 pp

Licence: MIT (repository code); Apache-2.0 (Qwen/Qwen3.5-4B base and unsloth/Qwen3.5-4B-GGUF Q4_K_M conversion)

Public accuracy: 70.1% · Sealed accuracy: 28.6% · Gap: 41.6 pp

Underlying model: Qwen3-Reranker-derived 4B cross-encoder

Licence: Apache-2.0

Public accuracy: 67.5% · Sealed accuracy: 25.6% · Gap: 41.9 pp

Underlying model: decision-machine-1 (weights not published)

Licence: proprietary API, closed weights

Public accuracy: 69.7% · Sealed accuracy: 24.4% · Gap: 45.3 pp

Licence: CC-BY-NC-4.0, research use only (XNLI and RACE in the training mix)

Public accuracy: 65.8% · Sealed accuracy: 29.2% · Gap: 36.6 pp

Underlying model: Phi-4-mini-instruct BF16

Licence: MIT

Public accuracy: 75.3% · Sealed accuracy: 29.5% · Gap: 45.8 pp

Underlying model: Qwen3.5-9B Base NVFP4 with compact BF16 decision head

Licence: Apache-2.0 entrant and upstream checkpoint

Public accuracy: 81.8% · Sealed accuracy: 28.6% · Gap: 53.2 pp

Underlying model: nvidia/diffusiongemma-26B-A4B-it-NVFP4

Licence: Apache-2.0 (repo and weights)

Public accuracy: 66.2% · Sealed accuracy: 22.4% · Gap: 43.8 pp

Underlying model: Qwen3-4B-Base + LoRA + learned pointer head; jaredpalmer/kev-4b

Licence: Apache-2.0

Public accuracy: 75.3% · Sealed accuracy: 26.0% · Gap: 49.4 pp

Underlying model: openbmb/MiniCPM5-2B with a trained LoRA adapter and pointer head (26.2M trainable parameters)

Licence: Apache-2.0 notices on the included code and the pinned base; the weights are an evaluation preview under EVALUATION-PERMISSION.md, not a cleared commercial release

Public accuracy: 78.4% · Sealed accuracy: 29.2% · Gap: 49.1 pp

Licence: Apache-2.0 (adapter code and weights; Qwen3.5-9B base is Apache-2.0)

Public accuracy: 99.1% · Sealed accuracy: 92.9% · Gap: 6.3 pp

Licence: proprietary API

Public accuracy: 86.6% · Sealed accuracy: 35.7% · Gap: 50.9 pp

Underlying model: Qwen3.8-27B through SimpleJev's direct-logit classifier

Licence: Apache-2.0 (Qwen weights); repository licence not stated

Public accuracy: 89.6% · Sealed accuracy: 34.1% · Gap: 55.5 pp

Underlying model: Qwen/Qwen3.8-Flash-Next, mixed NVFP4/FP8/INT4 NInfer artifact

Licence: Qwen Community License 1.0 (model); Apache-2.0 (engine)

Public accuracy: 88.7% · Sealed accuracy: 38.6% · Gap: 50.1 pp

Licence: patches Apache-2.0, shim MIT; weights under the Qwen licence (LICENSE-NOTICE.md)

Public accuracy: 99.6% · Sealed accuracy: 95.5% · Gap: 4.1 pp

Licence: proprietary API

Public accuracy: 74.0% · Sealed accuracy: 24.4% · Gap: 49.7 pp

Underlying model: Qwen/Qwen3.5-4B (frozen, BF16)

Licence: Apache-2.0 (code and weights)

Public accuracy: 74.9% · Sealed accuracy: 29.5% · Gap: 45.3 pp

Underlying model: Qwen/Qwen3.5-9B, frozen, per-option mean log-probability

Licence: no licence stated in the repository (public code); Apache-2.0 base weights

Public accuracy: 63.2% · Sealed accuracy: 25.6% · Gap: 37.6 pp

Underlying model: Qwen/Qwen3-0.6B-Base with a trained LoRA adapter and pointer head (10.6M trainable parameters)

Licence: Apache-2.0 notices on the included code and the pinned base; the weights are an evaluation preview under EVALUATION-PERMISSION.md, not a cleared commercial release

Public accuracy: 71.0% · Sealed accuracy: 24.7% · Gap: 46.3 pp

Underlying model: Qwen3.5-2B-Base with a trained decision readout, 1.9B

Licence: Apache-2.0

Public accuracy: 62.8% · Sealed accuracy: 33.1% · Gap: 29.7 pp

Underlying model: GLiFormer large (knowledgator/gliformer-large-v1, ~400M) behind a TypeSafe-compatible /v1/systemone server

Licence: MIT (code); GLiFormer weights per their model card

Public accuracy: 58.4% · Sealed accuracy: 30.8% · Gap: 27.6 pp

Underlying model: ModernBERT-large encoder + option-marker decision head, 421M, RLCD-trained

Licence: Apache-2.0

Public accuracy: 92.8% · Sealed accuracy: 45.8% · Gap: 47.1 pp

Licence: Gemma terms of use (Google) for the base weights; hosted Autoloops service

Public accuracy: 76.6% · Sealed accuracy: 26.6% · Gap: 50.0 pp

Licence: Apache-2.0 (jev-adapter server, LoRA and merged weights; base Ministral 3 8B Apache-2.0)

Public accuracy: 58.4% · Sealed accuracy: 25.0% · Gap: 33.4 pp

Underlying model: LiquidAI/LFM2.5-350M with a LoRA and a 6.5M-parameter pointer head (a kev clone)

Licence: Apache-2.0 (code); weights under LiquidAI's LFM1.0 licence, following the LFM2.5-350M base

Public accuracy: 85.3% · Sealed accuracy: 33.1% · Gap: 52.2 pp

Underlying model: Qwen3.6-35B-A3B

Licence: no licence file in the repository as of 2026-09-19; Qwen3.6 weights keep their own terms

Public accuracy: 57.1% · Sealed accuracy: 27.9% · Gap: 29.2 pp

Licence: Apache-2.0 code; Apache-2.0 weights (answerdotai/ModernBERT-large base)

Public accuracy: 83.1% · Sealed accuracy: 33.1% · Gap: 50.0 pp

Underlying model: Qwen/Qwen3.8-27B, NInfer NVFP4/FP8 artifact

Licence: Apache-2.0 (engine and submitted 27B artifact/base)

Public accuracy: 83.1% · Sealed accuracy: 33.1% · Gap: 50.0 pp

Underlying model: Qwen/Qwen3.8-27B, NInfer NVFP4/FP8 artifact

Licence: Apache-2.0 (engine and submitted 27B artifact/base)

Public accuracy: 71.4% · Sealed accuracy: 21.8% · Gap: 49.7 pp

Underlying model: Qwen3-8B-Base + LoRA + learned pointer head; jaredpalmer/kev-8b

Licence: Apache-2.0

Public accuracy: 89.6% · Sealed accuracy: 33.8% · Gap: 55.8 pp

Underlying model: Qwen3.6-35B-A3B BF16 with JevOne bidirectional option-logit mapping

Licence: Component-specific JevOne/Qwen/SGLang terms recorded in RESULT.md

Public accuracy: 77.9% · Sealed accuracy: 29.9% · Gap: 48.1 pp

Licence: Apache-2.0 (package and weights)

Public accuracy: 81.4% · Sealed accuracy: 28.2% · Gap: 53.1 pp

Underlying model: Qwen3.6-35B-A3B through SimpleJev's direct-logit classifier

Licence: Apache-2.0 (Qwen weights); repository licence not stated

Public accuracy: 66.7% · Sealed accuracy: 24.0% · Gap: 42.6 pp

Underlying model: Qwen3-0.6B-Base + LoRA + learned pointer head; jaredpalmer/kev-0.6b

Licence: Apache-2.0

Public accuracy: 68.4% · Sealed accuracy: 26.3% · Gap: 42.1 pp

Underlying model: Qwen3-8B BF16

Licence: Apache-2.0

Public accuracy: 71.9% · Sealed accuracy: 24.4% · Gap: 47.5 pp

Underlying model: Qwen/Qwen3-8B (frozen, BF16), the model of the author's demos

Licence: no licence file in the repository as of 19 Sep; Qwen3 weights Apache-2.0

Public accuracy: 53.2% · Sealed accuracy: 25.6% · Gap: 27.6 pp

Underlying model: MoritzLaurer/ModernBERT-large-zeroshot-v2.0 (~400M) used as a zero-shot NLI decision engine

Licence: Apache-2.0

Public accuracy: 86.1% · Sealed accuracy: 30.8% · Gap: 55.3 pp

Underlying model: Qwen/Qwen3.8-27B, frozen, read at the output head

Licence: Apache-2.0 (code); Apache-2.0 base weights

Public accuracy: 57.6% · Sealed accuracy: 27.9% · Gap: 29.7 pp

Underlying model: GLiClass ModernBERT-base fine-tuned decision model, 151M; v1.4 fixed inference engine

Licence: Apache-2.0

Public accuracy: 49.4% · Sealed accuracy: 27.3% · Gap: 22.1 pp

Underlying model: Qwen2.5-0.5B + LoRA + learned pointer head; jaredpalmer/kev-0.5b

Licence: Apache-2.0

Public accuracy: 79.7% · Sealed accuracy: 28.9% · Gap: 50.8 pp

Underlying model: bespokelabs/Bespoke-Nimble-9B (LoRA, adapter unchanged since 93ec5d6), merged into Qwen/Qwen3.5-9B@c202236 with the author's PEFT safe-merge

Licence: Apache-2.0 (weights); repository without a licence file as of 19 Sep

Public accuracy: 97.4% · Sealed accuracy: 89.0% · Gap: 8.4 pp

Underlying model: closed

Licence: proprietary API

Public accuracy: 55.4% · Sealed accuracy: 24.7% · Gap: 30.7 pp

Underlying model: GLiClass ModernBERT-base (knowledgator/gliclass-modern-base-v2.0) fine-tuned, 151M

Licence: Apache-2.0

Public accuracy: 54.1% · Sealed accuracy: 26.0% · Gap: 28.1 pp

Underlying model: Qwen3-1.7B BF16

Licence: Apache-2.0

Public accuracy: 87.0% · Sealed accuracy: 29.5% · Gap: 57.5 pp

Underlying model: Qwen3.8-27B, frozen, direct-logit readout averaged across two option orders

Licence: MIT code; Apache-2.0 Qwen weights

Public accuracy: 61.9% · Sealed accuracy: 23.4% · Gap: 38.5 pp

Underlying model: the author's own checkpoint model/jev1-2b-v2/checkpoint-45000-67.53 over a Qwen3.5-2B base, served by the author on his own machine; one request per decision returns a native probability for every offered option

Licence: not stated (the author published only an endpoint and the evaluation client)

Public accuracy: 87.4% · Sealed accuracy: 60.1% · Gap: 27.4 pp

Underlying model: google/diffusiongemma-26b-a4b-it, BF16; full generation with thinking enabled

Licence: Apache-2.0

Public accuracy: 56.7% · Sealed accuracy: 28.6% · Gap: 28.1 pp

Underlying model: GLiNER2 large schema-conditioned extractor (DeBERTa-v3-large encoder)

Licence: Apache-2.0

Public accuracy: 88.7% · Sealed accuracy: 42.2% · Gap: 46.5 pp

Underlying model: google/diffusiongemma-26b-a4b-it, BF16; OpenJev think=512

Licence: Apache-2.0

Public accuracy: 59.3% · Sealed accuracy: 34.7% · Gap: 24.6 pp

Underlying model: Qwen3.5-0.8B-Base fine-tuned as a JevLite decision model with per-question calibration

Licence: MIT code and training data; Apache-2.0 model weights

Public accuracy: 87.0% · Sealed accuracy: 38.6% · Gap: 48.4 pp

Underlying model: closed

Licence: proprietary API

Public accuracy: 52.4% · Sealed accuracy: 29.5% · Gap: 22.8 pp

Underlying model: microsoft/deberta-v3-large

Licence: Apache-2.0 (model card); DeBERTa-v3 keeps its own terms

Public accuracy: 60.6% · Sealed accuracy: 26.9% · Gap: 33.7 pp

Underlying model: MiniCPM5-2B-Base, 2.5B dense, with LoRA and native semantic decision heads

Licence: Apache-2.0

Public accuracy: 58.0% · Sealed accuracy: 29.2% · Gap: 28.8 pp

Underlying model: DeBERTa-v3-base schema-conditioned extractor (GLiNER2.5 boundary architecture), 194M

Licence: Apache-2.0

Public accuracy: 86.6% · Sealed accuracy: 35.1% · Gap: 51.5 pp

Underlying model: Qwen3.8-27B (Apache-2.0 base weights) with no additional fine-tuning; their own serving stack reads candidate-token scores in one forward pass per question and normalizes them into a decision probability, with no autoregressive decoding. The serving implementation is not public.

Licence: proprietary service over Apache-2.0 base weights (Qwen3.8-27B)

Public accuracy: 77.5% · Sealed accuracy: 29.9% · Gap: 47.6 pp

Underlying model: Qwen3.5-9B plus rank-8 LoRA and trained scalar decision head

Licence: MIT (loader); Apache-2.0 (adapter and pinned Qwen base); CC0-1.0 public training projection

Public accuracy: 64.5% · Sealed accuracy: 26.3% · Gap: 38.2 pp

Underlying model: Qwen3.5-2B plus rank-8 LoRA and trained scalar decision head

Licence: MIT (loader); Apache-2.0 (adapter and pinned Qwen base); CC0-1.0 public training projection

Public accuracy: 48.9% · Sealed accuracy: 32.8% · Gap: 16.1 pp

Underlying model: GLiNER2.5 multilingual schema-conditioned extractor, 287M

Licence: Apache-2.0

Public accuracy: 40.7% · Sealed accuracy: 24.0% · Gap: 16.7 pp

Licence: Apache-2.0 (code and CLM-v0.1-8B head); Qwen3-8B encoder Apache-2.0

Public accuracy: 54.5% · Sealed accuracy: 34.7% · Gap: 19.8 pp

Underlying model: Qwen3.5-0.8B through SimpleJev native assistant-prefill option-logit scorer

Licence: Apache-2.0 server; Apache-2.0 Qwen3.5-0.8B checkpoint

Public accuracy: 45.9% · Sealed accuracy: 28.6% · Gap: 17.3 pp

Underlying model: GLiNER2.5 small schema-conditioned extractor, 74M

Licence: Apache-2.0

Public accuracy: 48.1% · Sealed accuracy: 25.6% · Gap: 22.4 pp

Underlying model: Qwen3-0.6B BF16

Licence: Apache-2.0

Public accuracy: 53.7% · Sealed accuracy: 28.9% · Gap: 24.8 pp

Underlying model: intfloat/multilingual-e5-small (118M multilingual bi-encoder) fine-tuned on a typed-decision mix; each option is scored against the rendered state by cosine similarity, with temperature scaling and a conformal abstain set on top. One encoder pass per option, no tokens generated.

Licence: Apache-2.0 (verdictml code and the Manav2op/verdict-small checkpoint over intfloat/multilingual-e5-small)

Public accuracy: 97.8% · Sealed accuracy: 94.8% · Gap: 3.0 pp

Underlying model: DeepSeek-V4.1-Flash

Licence: open weights, proprietary API route

Public accuracy: 41.1% · Sealed accuracy: 9.1% · Gap: 32.0 pp

Underlying model: Mirror DeBERTa-v3-large 436M classification-span scorer

Licence: Apache-2.0 wrapper and Mirror release; upstream DeBERTa/model assets retain their own terms

Public accuracy: 37.2% · Sealed accuracy: 34.4% · Gap: 2.8 pp

Underlying model: 494M cross-encoder

Licence: Apache-2.0

Public accuracy: 39.4% · Sealed accuracy: 27.9% · Gap: 11.5 pp

Underlying model: 568M multilingual cross-encoder

Licence: Apache-2.0

Public accuracy: 33.8% · Sealed accuracy: 33.4% · Gap: 0.3 pp

Underlying model: 149M ModernBERT cross-encoder

Licence: Apache-2.0

Public accuracy: 31.6% · Sealed accuracy: 29.5% · Gap: 2.1 pp

Underlying model: ModernBERT-large with a per-option query/scoring head, ~400M parameters

Licence: MIT

Public accuracy: 84.4% · Sealed accuracy: 31.2% · Gap: 53.2 pp

Underlying model: google/diffusiongemma-26B-A4B-it BF16, one-step JSON canvas

Licence: MIT code; Apache-2.0 DiffusionGemma weights

Public accuracy: 85.3% · Sealed accuracy: 34.4% · Gap: 50.9 pp

Not ranked: runs on Jev (TypeSafe) — listed, not ranked

Underlying model: Jev (TypeSafe) behind classifier.dev's zero-shot classification API

Licence: MIT (code); hosted service

Public accuracy: 71.9% · Sealed accuracy: 21.8% · Gap: 50.1 pp

Underlying model: Qwen3.8-27B

Licence: open weights

Public accuracy: 22.1% · Sealed accuracy: —% · Gap: — pp

Not ranked: No completed sealed v1.4 run; retained as an unranked historical row

Underlying model: Needle 3 (121M parameters, 2-bit)

Licence: Apache-2.0 (model and package)

Public accuracy: 22.5% · Sealed accuracy: —% · Gap: — pp

Not ranked: No completed sealed v1.4 run; retained as an unranked historical row

Underlying model: Needle 3 (121M parameters, 2-bit)

Licence: Apache-2.0 (model and package)