On October 24, 2026, the arXiv preprint platform published a study titled “IFC-MCQ: Automatically Generating Multiple-Choice Benchmarks for Evaluating LLM Agents on IFC Data Retrieval,” which introduces the IFC-MCQ method—an automated technique for automatically constructing multiple-choice benchmarks from IFC files, used to evaluate the performance of large language model (LLM) agents in building information modeling (BIM) data retrieval tasks.
IFC is a widely used industry standard for exchanging building information in the architecture, engineering, and construction (AEC) sector, and it is increasingly being queried by LLMs via natural language. However, current evaluation methods rely on free-text responses, which require significant human effort or additional LLMs for scoring, making large-scale deployment challenging. IFC-MCQ enables precise, automated scoring by constructing multiple-choice questions with clearly defined answer options: its taxonomy encompasses 15 categories and 36 subtypes across four major groups; correct answers are derived directly from the IFC file itself, while distractors are generated from values extracted from the file or based on predefined rules.
The research team applied IFC-MCQ to IFC models of varying scales and modeling characteristics, validating its generation coverage and testing two LLM agent configurations—tool-based and code-based retrieval—on selected models. The results revealed failure patterns within specific categories, with performance varying according to differences in the source IFC model and the retrieval interface. The paper notes that IFC-MCQ provides a scalable benchmarking capability for LLM‑driven IFC retrieval in AEC workflows.