Skip to content
AI NEWSSee every step of AI
Bookmarks
THE JOURNAL
AI NEWS JOURNAL
ONE EVENT. MANY PERSPECTIVES.

Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content Moderation

arXiv:2610.33258v1 Announce Type: new Abstract: Content moderation compresses a graded, contested human judgment into a binary act (leave up / take down) over billions of items, at a scale where running a frontier LLM on every post is uneconomical. A new class of frozen "System One" models returns typed, calibrated decisions instead of text at ~USD 0.000025 per call and ~100 ms. We run the first independent audit of one such model on 142,658 human-labeled comments, and benchmark it against 14 frontier reasoning models (the GPT-5.x, Claude, Gemini 3, and Grok families) on a shared probe set. Two findings stand out. First, the cheap model is accuracy-competitive: its zero-shot classification AUROC (0.83) is not statistically separable (paired bootstrap, n=56; a failure to reject parity, not

1 independent sources · 1 reportsOldest first ↕
Newest first

arXiv cs.CL

Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content Moderation

arXiv:2610.33258v1 Announce Type: new Abstract: Content moderation compresses a graded, contested human judgment into a binary act (leave up / take down) over billions of items, at a scale where running a frontier LLM on every post is uneconomical. A new class of frozen "System One" models returns typed, calibrated decisions instead of text at ~USD 0.000025 per call and ~100 ms. We run the first independent audit of one such model on 142,658 human-labeled comments, and benchmark it against 14 frontier reasoning models (the GPT-5.x, Claude, Gemini 3, and Grok families) on a shared probe set. Two findings stand out. First, the cheap model is accuracy-competitive: its zero-shot classification AUROC (0.83) is not statistically separable (paired bootstrap, n=56; a failure to reject parity, not

Timeline and related coverage