Search headlines, summaries and articles to connect the dots.
Jev ranks warehouse tables and labels agent failures in under a second, but adding retrieval hints leaves accuracy flat across 20 models on the Agentic Analytics Benchmark.
Microsoft and Hugging Face have jointly launched the open-source framework ThinkingBox, which generates binary 0/1 rewards by comparing the LLM agent’s final database state after execution with the expected state, for reinforcement learning-based fine-tuning. The accompanying benchmark, ThinkingBox-Bench, includes 507 real-world backend tasks, none of which were used during training. After 50 steps of GRPO fine-tuning, Qwen3.8-27B achieved a 9.08-percentage-point improvement in pass@1 on the held-out test set, with Observed 20/20 nearly tripling; however, pass@20 declined in some domains, reflecting a trade-off between breadth and stability.
We added GPT-6 Astra, Luna and Sol, Claude Opus 5.5 and Claude Sonnet 5.5 to the Agentic Analytics Benchmark: all five land mid-board within a point of each other, and Sonnet 5.5 brings the biggest change in turns, cost and truncated runs.
