Abstract:Language models are widely used as synthetic respondents in surveys, behavioral experiments, and market research. Most of these respondents read text, whereas shoppers in a store choose by looking at shelves, which makes vision-language models (VLMs) natural synthetic shoppers. We test whether VLM shoppers follow real consumers at two levels. At the purchase level, VLM shoppers built from household records plan their trips from a store directory and select products from shelf images, and we compare their purchases with six public datasets from three countries. The synthetic baskets hit held-out purchases as often as the households' own past trips, and synthetic group differences correlate with real ones twice as strongly as those of a trained regression. Without fitting, VLM shoppers reproduce alcohol purchases by gender and age in Russia and trip sizes by store distance in Taiwan, and respond to situations, prices, and displays within measured ranges. At the level of a single shelf choice, a model's answer confounds the product with its position and letter label. We therefore introduce ShelfSight, shelf renderings that vary products, positions, labels, viewpoints, and prices independently, together with a factor model that separates product preferences from the effects of label and position, which we call answer effects. On ShelfSight, hosted VLMs read the shelf, follow stated needs, keep their product preferences across views, and respond to price as budget shoppers. These answer effects, a preference for the letter A and for positions in the image, are estimated once per image type and removed. VLMs are thus good synthetic shoppers whose corrected shelf choices reflect product preferences. Submission history Access Paper: View PDF HTML (experimental) TeX Source Additional Features Audio Summary Current browse context: References & Citations Google Scholar Semantic Scholar Bookmark Institution arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
AI NEWS JOURNAL
arXiv cs.CVAI NEWS / JOURNAL
Seeing Like a Shopper: Validating Vision-Language Models as Synthetic Consumers
arXiv:2611.09977v1 Announce Type: new Abstract: Language models are widely used as synthetic respondents in surveys, behavioral experiments, and market research. Most of these respondents read text, whereas shoppers in a store choose by looking at shelves, which makes vision-language models (VLMs) natural synthetic shoppers. We test whether VLM shoppers follow real consumers at two levels. At the purchase level, VLM shoppers built from household records plan their trips from a store directory and select products from shelf images, and we compare their purchases with six public datasets from three countries. The synthetic baskets hit held-out purchases as often as the households' own past trips, and synthetic group differences correlate with real ones twice as strongly as those of a trained regression. Without fitting, VLM shoppers reproduce alcohol purchases by gender and age in Russia and trip sizes by store distance in Taiwan, and respond to situations, prices, and displays within measured ranges. At the level of a single shelf choice, a model's answer confounds the product with its position and letter label. We therefore introduce ShelfSight, shelf renderings that vary products, positions, labels, viewpoints, and prices independently, together with a factor model that separates product preferences from the effects of label and position, which we call answer effects. On ShelfSight, hosted VLMs read the shelf, follow stated needs, keep their product preferences across views, and respond to price as budget shoppers. These answer effects, a preference for the letter A and for positions in the image, are estimated once per image type and removed. VLMs are thus good synthetic shoppers whose corrected shelf choices reflect product preferences.