CORTEXA
← Browse
arxivcs.AI2026-07-12

Imaging-101: Benchmarking LLM Coding Agents on Scientific Computational Imaging

Siyi Chen, Jiahe Ying, Yixuan Jia, Yuxuan Gu, Enze Ye, Weimin Bai, Zhijun Zeng, Shaochi Ren, Binhong Gao, Yubing Li, Tianhan Zhang, He Sun

Computational imaging, which recovers hidden signals from indirect, noisy measurements, underpins quantitative discovery across scientific disciplines, yet building a correct reconstruction pipeline demands deep domain expertise and remains laborious even for domain scientists. We introduce Imaging-101, a benchmark of 57 expert-verified computational imaging tasks spanning six scientific domains, each grounded in a peer-reviewed paper and canonicalized into a standardized four-stage pipeline (preprocessing, forward physics modeling, inverse solver, and visualization) Three evaluation tracks (planning, function-level unit tests, and end-to-end reconstruction) probe distinct agent capabilities across the full pipeline. Evaluating seven frontier LLMs uncovers systematic challenges in applying coding agents to computational imaging that go beyond those exposed by general coding benchmarks, spanning algorithm selection, physical convention handling, and pipeline integration. These findings highlight concrete capability gaps and point toward skill-augmented, domain-specialized agents as a practical path to reliable computational imaging assistance.

View free PDFSource page

Related papers

arxivcs.AI2026-07-08

Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

Ethan Chung, Chuanjun Zheng, Jasper Tan, Jingxi Li, Haopeng Zhang, Huaijin Chen

Vision-language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging. We present ImagingBench, a benchmark of 20 computational imagi…

View free PDFSource page
arxivcs.CRcs.AIcs.LG2026-07-20

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

Devina Jain, David Hartmann, Chuan Li

LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adap…

View free PDFSource page
arxivcs.CRcs.AIcs.SE2026-07-22

IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

Ankur Singh, Jinqiu Yang, Tse-Hsun Chen

AI coding agents powered by LLMs are increasingly integrated into real-world software development, where they generate, edit, and execute code with autonomous access to local files and tools. Coding agents inherit security risks from both the LLM backbone, where adversarial promp…

View free PDFSource page