CORTEXA
← Browse
arxivcs.CVcs.HC2026-07-22

MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction

Yue Zhao, Hongxu Liu, Feiyu Wang, Xiaoyu Yang, Tong Ge, Zhen Yang, Chao Wang, Qiong Zeng

Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations mainly focus on single-chart generation and overlook coordinated multi-view interface construction, which requires joint reasoning about data semantics, view coordination, and interaction logic. Consequently, MLLM capabilities in this setting remain underexplored, and the field lacks a dedicated benchmark for systematic assessment. We introduce MV-Bench, a benchmark for evaluating MLLMs on coordinated multi-view interface construction. Instead of relying on incomplete or inconsistent open-source implementations, we use Tableau workbook files as ground truth because they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web interfaces through structured intermediate representations. The benchmark contains 92 base interfaces and 1,048 verified instances created by recombining chart types, datasets, and interaction patterns. Each instance includes executable code, a rendered interface, a dataset, and interaction annotations. We evaluate five state-of-the-art MLLMs in a single-pass setting using metrics for visual fidelity, data binding correctness, and interaction completeness. The strongest model achieves 75.45 percent accuracy in visual layout reproduction, but only 21.71 percent in data binding and 11.68 percent in interaction completeness. These results show that current MLLMs can reproduce visual appearance but remain limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Iterative refinement improves code executability but does not substantially reduce the gap in data binding and interaction generation.

View free PDFSource page

Related papers

arxivcs.CLcs.AIcs.CVcs.HC2026-07-31

FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models

Jeffrey M. Girard, Jason Z. Zheng, Jacqueline R. Vertino, Antony D'Avirro, Benjamin Peloquin

Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same…

View free PDFSource page
arxivcs.CVcs.HC2026-06-30

AA: A Multi-view Multimodal Dataset for Screen-based Gaze Estimation

Chang Liu, Jiaqi Liu, Zhoutong Ye, Xinjie Shen, Chun Yu, Yuanchun Shi

We present AA, a multi-view multimodal dataset for screen-based gaze estimation. The dataset captures synchronized facial observations from eight fixed screen-mounted cameras and two additional side-view cameras, paired with precise screen-space gaze targets collected under contr…

View free PDFSource page
arxivq-bio.NCcs.AIcs.CVcs.HC2026-07-03

Modeling the Impact of Visual Brand Language on Attention, Object Recognition, and Memory Retrieval

Rachel F. Heaton, John E. Hummel

Visual brand language is the set of visual properties that convey brand identity for a product. What is the impact of visual brand language on a person's ability to recognize and understand the functional identity of an object? Using an empirically supported modeling framework ba…

View free PDFSource page
arxivcs.AIcs.CVcs.HC2026-07-10

Exploring Agentic Workflows for Generating High Quality Math Visual Aids

Rizwaan Malik, Ashna Khetan, Isabel Sieh, Samin Khan

Mathematical diagrams play a crucial role in K 12 education, both as problem components and as scaffolding for student comprehension. However, current AI tools, including Large Language Models (LLMs), struggle to reliably generate accurate and pedagogically sound visual diagrams,…

View free PDFSource page
arxivcs.CVcs.HC2026-07-17

Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs

Maeve Hutchinson, Abderrahmane Wassim Mehdaoui, Pranava Madhyastha

Understanding how vision-language models (VLMs) interpret data visualizations remains an open problem, and is increasingly important as these models are used for analytical tasks where reliable reasoning is essential. We introduce a lightweight, diagnostic saliency map method tai…

View free PDFSource page
arxivcs.CVcs.HC2026-07-06

PAGE: Towards Practical Human-level Gaze Target Estimation

Zhoutong Ye, Chengwen Zhang, Zhaibin Cui, Mingze Sun, Jiaqi Liu, Xiangwu Li, et al.

Gaze target estimation, the task of predicting where a person is looking in a scene, is crucial to understanding human attention and intent. It is a challenging task that combines high-level understanding of global scene semantics and precise spatial reasoning using human appeara…

View free PDFSource page