본문으로 건너뛰기
AI NEWS JOURNAL
Hacker News Show HNAI NEWS / JOURNAL

Show HN: System 1.5 Model – Inkling-2-Mini Decision

Article URL: https://huggingface.co/eltokh7/inkling-2-mini-decision Comments URL: https://news.ycombinator.com/item?id=50595931 Points: 3 # Comments: 0

번역 서비스가 일시 중지되어 원문을 표시합니다.

An open-weights System 1.5 adaptive-depth decision model built on thinkingmachines/Inkling-2-Mini (26.4B total / 3.4B–5.4B active MoE, 24 physical layers with a 12-layer looped core). Instead of generating prose or JSON tokens, it evaluates a state and typed questions (choice, noul, score) in a single forward pass and dynamically decides how many internal loops (1, 2, or 3 = 24, 36, or 48 logical layers) to spend before answering — with zero reasoning tokens. Key Results Routine decisions (decision-v7/test, $n=1{,}440$): 85.56% accuracy, 0.1944 Brier, 0.0235 ECE — right alongside 27B-dense Kev-27B (86.3%) at ~5× lower active prefill FLOPs. Difficulty-routed early exit (τ = 0.90): 54.2% of routine decision-v7 questions exit right after Loop 1 (only 8.8% of hard OOD hard-v1 questions do — 76.8% automatically route to Loop 3). Real GPU wall-clock speed (merge_and_unload + torch.compile, RTX PRO 6000 Blackwell, median headline / mean in parens): Batch-1 Early Exit (τ = 0.90): 81.4 ms median (97.4 ms mean) — 1.68× faster median than compiled Loop 3 (136.8 ms median / 146.3 ms mean) with −0.07 pp accuracy difference (85.49% vs. 85.56%). Batch-1 Early Exit (τ = 0.85): 79.6 ms median (95.4 ms mean) — 1.72× faster median. Per-pass medians (batch-1): 65.6 ms (Loop 1) / 100.2 ms (Loop 2) / 136.8 ms (Loop 3). Batched throughput (bs = 8, causal_conv1d): 27.9 ms/q (35.8 q/s) at Loop 1 and 61.4 ms/q (16.3 q/s) at Loop 3. Latent reasoning on untrained OOD hard tasks (hard-v1): Going from Loop 1 → Loop 3 adds +26.8 pp on hard-v1/dev (39.9% → 66.7%) and +24.3 pp on hard-v1/test (42.6% → 66.8%), beating 9B-dense Kev-9B v1 (58.2% / 58.4%) by +8.4 pp with 5.4B active parameters. Architecture Frozen 26.4B MoE + Attention LoRA (r = 32): LoRA adapters (17.5M params) attached to self_attn.{q,k,v,o}_proj.{linear,gate_proj}; lm_head and vision_tower dropped for text decisioning. Zero-Waste Layer-17 Halting Gate (CoreGateHead, 1.2M params): After Loop 1 or Loop 2, a lightweight bilinear probe (~0.05 ms) reads the Layer-17 core output before running exit layers 18–23. If calibrated confidence clears threshold, it skips the remaining loops and runs exit layers once. Two-Stage JointSchemaHead (6.0M params): Stage 1 (Evidence Routing): and every token cross-attend to 8 summary tokens of the input state. Stage 2 (Cross-Option Attention): Options and self-attend to compare directly before bilinear pointer scoring. Multi-Loop Deep Supervision: Trained jointly across Loops 1, 2, and 3 (0.2·L1 + 0.3·L2 + 0.5·L3) using Label-Smoothed Cross-Entropy + Brier Loss (0.5), then temperature-calibrated on decision-v7/calibration. Quickstart from modeling import InklingDecisionModel model = InklingDecisionModel.from_pretrained("eltokh7/inkling-2-mini-decision") out = model.decide( state="Customer email: 'My card was charged twice for order #4821 yesterday. Please refund.'", questions={ "route": { "type": "choice", "instructions": "Which team should handle this ticket?", "criteria": { "billing": "Duplicate charges, refunds, invoices", "tech": "Bugs, login issues, crashes", "sales": "Pricing, upgrades, quotes", }, }, "urgent": { "type": "noul", "instructions": "Does this require immediate escalation?", }, }, threshold=0.90, # set to 1.01 to always run all 3 loops ) print(out) Benchmarks 1. Loop 1 vs. Loop 2 vs. Loop 3 2. Zero-Waste Early Exit & Real CUDA Latency (v7/test, 60 questions, avg 223 tokens, RTX PRO 6000 Blackwell) Headline numbers are median (p50) latency to match Kev’s benchmark convention (mean in parens). | Mode (Merged LoRA + torch.compile, max |Δp| = 3.5e-4) | v7/test Acc | v7/test L1 Exit | hard-v1/dev L1 Exit | Median (p50) | Mean | Speedup vs. L3 (p50) | |---|---:|---:|---:|---:|---:|---:| | Fixed Loop 3 (48L) | 85.56%* | 0.0% | 0.0% | 136.8 ms | 146.3 ms | 1.00× | | Fixed Loop 2 (36L) | 84.93% | — | — | 100.2 ms | 107.0 ms | 1.37× | | Fixed Loop 1 (24L) | 82.29% | 100.0% | 100.0% | 65.6 ms | 70.1 ms | 2.09× | | Early Exit (τ = 0.90) | 85.49% | 54.2% | 8.8% | 81.4 ms | 97.4 ms | 1.68× | | Early Exit (τ = 0.85) | 85.35% | 60.0% | 13.9% | 79.6 ms | 95.4 ms | 1.72× | | Batched (bs = 8, causal_conv1d, L1 / L3) | — | — | — | 27.9 / 61.4 ms/q | — | 35.8 / 16.3 q/s | *Re-verified on full decision-v7/test ($n=1{,}440$) after merge_and_unload() + causal_conv1d: 86.04% (1,239 / 1,440). Limitations & Negative Results Extrapolating past Loop 3: Running 4 or 5 loops zero-shot degrades accuracy (63.5% → 59.5% → 46.1% on hard-v1/dev) because Inkling-2-Mini was pre-trained with fixed num_passes = 3. Raw date arithmetic: Without deterministic date_facts preprocessing, multi-way date subtraction (hard_temporal_numeric) remains harder than multi-hop or tradeoff reasoning. License: Apache 2.0 (inherited from thinkingmachines/Inkling-2-Mini). Model tree for eltokh7/inkling-2-mini-decision Base model

자료 출처

원문 읽기 ↗

관련 주제

← 전체 소식 이어 읽기