Picture guiding a robot arm by hand while its motors supply just enough torque to support its weight and keep the robot upright. This simple interaction, a...
Article URL: https://github.com/s-nish/desk Comments URL: https://news.ycombinator.com/item?id=50430522 Points: 1 # Comments: 0
Eval-driven development helps you build AI agents against a consistent set of metrics. Define good behavior, write the eval first, then compare experiments.
arXiv:2611.08683v1 Announce Type: new Abstract: Large language models (LLMs) have been evaluated on their ability to identify emotions expressed in text and produce fluent emotional language. However, output predictions alone do not reveal how fine-grained emotion distinctions are organized internally. We study seven LLMs on GoEmotions, an English Reddit dataset annotated with 27 fine-grained emotion categories plus neutral. We compare model predictions and emotion-prompted final-token representations with the 28 GoEmotions categories and with coarser emotion-family and sentiment groupings. We find a consistent pattern of $\textit{overcompression}$ relative to the fine-grained GoEmotions taxonomy. Under the tested KMeans protocol, when we cluster their hidden representations without using emotion labels, the clearest split is into two groups that largely separate positive from negative emotion. Coarser emotion groupings are also recovered with fewer clusters than the full 28-category taxonomy. Yet linear probes retain substantial fine-grained predictive information. These results suggest that fine-grained emotion information remains accessible even though broader affective structure more strongly organizes the unsupervised geometry.
arXiv:2611.08647v1 Announce Type: new Abstract: Intra-weight injection lets one model change another model's behaviour by editing its weights directly, without gradients. Such edits can range from altering lexical associations to enforcing output formats or updating factual knowledge. Despite growing interest in these approaches, there has been no systematic way to evaluate how well models perform them. We introduce IWIBench, which measures a large editor model's ability to modify the weights of a smaller target model (typically Gemma 3 4B IT) to achieve one of ten classical text-editing objectives while preserving its other capabilities. Editors are limited to four tools for reading and writing weights, with bounded tensor access and a fixed budget for generating and inspecting rollouts. A validator applies each proposed patch in a clean environment and compares the edited model against the unmodified target. Evaluating frontier closed- and open-weight models reveals sharply diverging capabilities, both across models and across tasks. A small number of models, most notably Astra and GPT-6.1 Sol, perform far ahead of the rest. The recency of this ability (little of it is evident in GPT-5.6 Sol, for instance), together with qualitative analysis, suggests it has been acquired through recent training. Tasks also vary widely: structured response resisted every attempt, and answer-length control yielded only one successful solution, whereas alias acquisition and recoding proved most amenable. Success in one task family correlated poorly with success in others, indicating an early and uneven capability frontier. We discuss strategies that editors repeatedly adopted and conclude with an outlook on intra-weight injection as an emerging but already potent approach to gradient-free behaviour modification.
Article URL: https://github.com/infrixo-systems/rag-evaluation-starter Comments URL: https://news.ycombinator.com/item?id=50422943 Points: 1 # Comments: 0
Letting coding agents (such as Claude Code) work autonomously on repos on a daily-basis, brings a high level of uncertainty. For those who won't let the agent decide alone whether it can create/modify or delete any folder on their repos; I wanted to be able to trace actions performed and being able to let the agent continue his work autonomously, or ask him to request a human approval according to some criteria defined. After investigations, the cleanest way to do it was according to me to develop a deterministic control layer between the LLM and the work of the agent that would track and catch potential non-respect of pre-defined rules. How? by first setting up approval rules (Pass/requires approval/refused) before the work of the agent, and afterwards controlling cryptographic SHA 256 verifications, etc. This allows to compare the before-after work of the agent. I have put in my public GitHub repo a demo that you can launch easily and observe different scenarios behaviors like applied authorized modification, protected file waiting for Human approval, or outside perimeter authorization modification refused. The demo is offline, does not involve the official runtime, in native node.js and without dependencies. It should still allow you to understand the core principles. I would like to know: how do you implement and handle safety boundaries? Especially if you let agents work on critical data/sensitive codebases? With which guardrails? Comments URL: https://news.ycombinator.com/item?id=50422833 Points: 1 # Comments: 0
Hello HN!I've always had a bad time redistributing programs that use shared libraries, AppImage in my opinion does too much, and a container is an overkill - not always integrates smoothly.I've moved to nix since long ago, but redistributing between nix and other distros or vice versa it's still a pain in the ass, more so if you want to run an alien app in nix or a nix app without the package manger in any other distro.For that, with the help of Claude, I've implemented a format that deduces at pack-time all the shared libraries a program needs, puts them, together with its own glibc and a patched dynamic linker into a single executable.This means that you don't actually need a runtime nor any special capability, it can run inside a FROM SCRATCH docker, alpine linux etc... And it also supports coreutils-like multiple programs accessible via symlinks, so it is a good fit for that purpose.You can just use:> felf pack $(which git) --output git.felfand you'll have already a git binary that works everywhere, no matter which distro you are building this from. This only packs the binary and their dependencies.Essentially, the effect is similar as if you LD_PRELOAD's all the libraries that it requires.In the end, we have been fighting these demons since long long ago, chroots/docker/appimage/flatpack/nix etc...So this xkcd applies: https://xkcd.com/927/I'm happy to share the project, I hope it inspires new discoveries/tools in this regard! Comments URL: https://news.ycombinator.com/item?id=50415219 Points: 2 # Comments: 0
Article URL: https://github.com/voidixyn/glassbrain Comments URL: https://news.ycombinator.com/item?id=50423154 Points: 1 # Comments: 1
Article URL: https://github.com/alam00000/bentopdf-hyper-compress Comments URL: https://news.ycombinator.com/item?id=50416782 Points: 1 # Comments: 0
