On October 24, 2026, a paper titled “baseVultra: Compact Nonlinear Recurrence with Reallocated Capacity” was first published on the arXiv platform (arXiv:2610.30154v1), introducing a novel compact nonlinear recurrence unit called baseVultra. This work addresses the current paradigm of gated recurrent units, which is still dominated by LSTMs and GRUs, by exploring the reallocation of recurrent computational resources from gate transformations to the candidate value generation pathway.
The core design of baseVultra includes: using a single H×H dense matrix dedicated to candidate generation; replacing the dense recurrent matrix in traditional gating with a diagonal interaction at O(H) complexity; integrating candidate updates via a dense branch, two low-rank auxiliary branches, an EMA cache, and a learnable output mixture; and introducing a lightweight second-order diagonal access to the hidden states h_{t-2} from the previous two steps in the gating module. On the TinyShakespeare character-level modeling task, its 200K-parameter single-layer configuration achieves a validation loss of 1.594, outperforming LSTMs (1.604) and GRUs (1.616) with the same number of parameters, while the computational cost of dense matrix–vector operations is approximately 54% of that of GRUs.
On the larger-scale text8 dataset, baseVultra achieves 1.854 bits per character (bpc), slightly trailing GRU (1.849) and LSTM (1.837); meanwhile, on the formal language probe Counter task (with hidden dimension H=16), its out-of-distribution (OOD2) accuracy reaches 0.991, significantly surpassing GRU (0.761) and LSTM (0.691). The paper notes that such gains are highly dependent on a compact configuration and diminish as the hidden dimension increases, indicating that baseVultra is well-suited for resource-constrained streaming sequence modeling scenarios.