Abstract:We introduce Nemotron 3.5 Bolt 74B-A9B 24GB, a hybrid Mamba-Transformer vision-language model that optimizes speed and intelligence per GB of memory used for local agentic workloads. Derived from the Nemotron 3.5 Super VL 120B-A12B model family, Nemotron 3.5 Bolt 74B-A9B 24GB combines Star Elastic compression with 2-bit Quantization-Aware Distillation for memory-constrained deployment. The model is optimized for reasoning, agentic tool use, and web & terminal interactions. We also trained a LiLiCorr speculative drafter that is released alongside the model to reach our target inference speed. The resulting model achieves a higher accuracy than leading models with similar memory footprints and speed across many benchmarks for general agentic capabilities (e.g., GDPVal v2), web browsing (e.g., BrowseComp), and terminal use (e.g., Terminal-Bench 2.1), with a relatively high TPS of 84 on RTX Spark that is comparable to models with much lower active parameter counts and more than sufficient for long-horizon agentic applications.
Submission history
Access Paper:
View PDF
HTML (experimental)
TeX Source
Additional Features
Audio Summary
Current browse context:
References & Citations
Google Scholar
Semantic Scholar
Bookmark
Institution
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.