A newly published arXiv study (2610.30997v1, posted on October 24, 2026) points out that, in extraction tasks where language models are required to output JSON conforming to a specified schema, if the schema’s `minItems` field is set to a certain minimum number of items—for example, `minItems: 3`—and the input document actually contains fewer corresponding entities than that threshold, most open-source large models do not simply return the shorter list as is. Instead, they proactively “pad” the list to meet the minimum requirement, generating redundant yet seemingly reasonable entries.
The research team evaluated eight open-source weighting models on 558 real-world documents (from nine annotated datasets), using six hosted-service models as a baseline. The results show that although the “minimum number of items” is only a hint in the schema and not enforced, it increases the proportion of output lists whose length exceeds the true count from 34.5% to 55.4%. Moreover, the filled-in content largely consists of text fragments already present in the documents, with about 50% passing the “each item appears in the original text” verification, indicating strong concealment. In addition, common correction strategies—such as providing natural-language instructions (e.g., “at most k items”), adding an “N/A” option, or re‑asking questions based on validation errors—failed to effectively curb the filling behavior; some even exacerbated under‑detection or over‑generation.
The study emphasizes that such padding behavior is not a random hallucination, but rather a systematic bias jointly induced by schema design and the model’s response mechanism, posing potential risks to downstream applications that rely on precise, structured outputs—such as knowledge graph construction and compliance report generation. The authors have open-sourced all experimental code, answer lists, and run logs (https://github.com/AbhishekBarali/padding-to-k），供社区复现与改进。.