CORTEXA
← Browse
arxivcs.AIcs.CVcs.LGcs.MM2026-07-18

From Modalities to Propositions: A Language-Centric Framework for Multimodal Intelligence

Nadine Chang, Maying Shen, Shizhe Diao, Jialiang Wang, Jingde Chen, Thomas Breuel, Pavlo Molchanov, Rafid Mahmood, Jose M. Alvarez

We propose a language representation for multimodal data in which any observation, whether image, video, or text, is expressed as a bag of atomic propositions, simple statements about the entities, actions, and relations in a scene. A global semantic codebook unifies these into a shared vocabulary of canonical atomic propositions, placing every modality and observation into one interpretable space that spans fine grained facts to high level concepts and composes into richer ones. This brings interpretability with reasoning, cross-modal understanding and retrieval, and compositionality that enables complex multimodal understanding, rich data curation and complex structured retrieval. We demonstrate the framework on autonomous driving and open-world data.

View free PDFSource page

Related papers

arxivcs.CVcs.AIcs.LG2026-07-31

TAVI-TEC: An AI-Based Tool for Procedural Planning of Transcatheter Aortic Valve Implantation

Alessandra Zerillo, Stefano Cannata, Diego Bellavia, Daniele Ciriello, Simone Manini, Salvatore Pasta, et al.

Computed tomography angiography (CTA) is crucial for preprocedural TAVI planning, providing the anatomical information required for prosthesis sizing and vascular access assessment. As the volume of TAVI procedure increases, improving efficiency and standardizing annotations is b…

View free PDFSource page
arxivcs.LGcs.AIcs.CV2026-07-31

A Human-Centered Validation of the Explainability-Performance Coefficient

Christian Oliva, Luis F. Lago-Fernández

The rapid adoption of deep learning models in high-risk domains has intensified the need for trustworthy Explainable Artificial Intelligence (XAI). However, objectively evaluating explanation fidelity and aligning XAI metrics with human-centered understanding remain critical open…

View free PDFSource page
arxivcs.CVcs.AIcs.LG2026-07-31

Have I Seen You? Embedding Behavior Signals Synthetic Face Dataset Membership

Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo, Jacques Klein, Tegawendé F. Bissyandé

Synthetic face datasets are increasingly used to reduce privacy exposure and data access constraints in biometric recognition. Yet the generators that produce these datasets are trained on real faces, so synthetic data may still reveal their real source data. We study this risk t…

View free PDFSource page
arxivcs.LGcs.AIcs.CV2026-07-31

SERUM: State Extraction and Refinement for User Modeling

Andy J. Phu, James Mooney, Karin de Langis, Khanh Chi Le, Dongyeop Kang

Agentic assistants capable of proactive, personalized interactions require structured models of user intent and workflow. However, building these models from raw, unstructured screen activity remains an open challenge. We present SERUM, a multi-pass framework that extracts finite…

View free PDFSource page
arxivcs.HCcs.AIcs.LGcs.MM2026-07-24

Unboxing Diffusion Models for the Arts: Interactive Model Bending and Practice-Based Explainability

Ahmed M. Abuzuraiq, Philippe Pasquier

Explainable AI (XAI) in creative practice can be less about technocentric explanation and more about enabling artists to inspect modify and debug models as part of making Yet largescale texttoimage diffusion systems are typically presented as opaque endtoend tools limiting this k…

View free PDFSource page
arxivcs.CVcs.AIcs.LG2026-07-24

CARDIAG: A Dense Segment Classification Benchmark of Deep Learning Architectures for Coronary Angiography

Dominik Bernard Lau, Hubert Malinowski, Jerzy Szyjut, Adam Brzeski, Tomasz Dziubich, Radosław Targoński, et al.

Accurate pixel-level classification of coronary angiograms is critical for cardiovascular disease assessment, yet the field lacks standardized evaluation protocols. In this work we demonstrate a new benchmark for the assessment of deep learning models which densely classify pixel…

View free PDFSource page