Abstract:Recent work has hypothesised that large language models may represent some concepts as multi-dimensional manifolds instead of single directions. Using these manifolds to interpret or steer representations often assumes that moving along them corresponds to a continuous change in the represented concept. In this work, we test this continuous manifold assumption on the prototypical weekday manifold. Despite its striking circular geometry, we find that the weekday representations behave more like seven discrete states arranged in a circle than a continuous ring. First, we show that local sensitivity has seven minima aligned with the weekdays, producing a step-like mapping between representation and behaviour under on-manifold steering. Second, we test whether any concepts are represented along the manifold outside the seven discrete states, and find neither time of day, uncertainty, nor natural text populate the intermediate space. Together, these results challenge the continuous interpretation of the weekday manifold and motivate testing whether movement along other representation manifolds correspond to meaningful changes in model representations and behaviour.
Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) Cite as: | arXiv:2610.37796 [cs.LG] | (or
arXiv:2610.37796v1 [cs.LG] for this version) | https://doi.org/10.48550/arXiv.2610.37796
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
Access Paper:
View PDF
HTML (experimental)
TeX Source
Additional Features
Audio Summary
Current browse context:
References & Citations
Google Scholar
Semantic Scholar
Bookmark
Institution
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.