CORTEXA
← Browse
arxivcs.LG2026-07-06

A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

Nima Eshraghi, Lovedeep Gondara, Yuqing Huang, Sagarika Suresh, Leizer Teran, Jithin Pradeep, Xiaotong Xu, Fanny Chevalier

Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability $p$, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the dense-steering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.

View free PDFSource page

Related papers

arxivcs.LGcs.SE2026-07-31

Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies

Jan Marius Stürmer, Jascha Knack, Tobias Koch, Andreas Weinmann

Large language models (LLMs) have demonstrated a strong ability to generate syntactically correct code from natural-language specifications. In this study, we explore how LLMs can be harnessed to automatically translate a neutral graph representation of fluid system models into e…

View free PDFSource page
arxivcs.ARcs.LG2026-07-30

LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference

Sangjin Kim, Yuseon Choi, Jungjun Oh, Byeongcheol Kim, Hoi-Jun Yoo

As large language models (LLMs) continue to demonstrate exceptional capabilities across various domains, the challenge of achieving energy-efficient and accurate inference becomes increasingly critical. This work presents LightRot, a lightweight rotation scheme and dedicated hard…

View free PDFSource page
arxivcs.LGcs.AI2026-07-31

MOT-SR: Multi-Objective Tool-Augmented Scientific Equation Discovery with Large Language Models

Boxiao Wang, Runxiang Wang, Kai Li, Chongming Li, Zhiwei Chen, Yifan Zhang, et al.

Symbolic Regression (SR) aims to discover analytical equations from observational data and plays a central role in scientific modeling. While recent Large Language Model (LLM) based approaches show promise, they face two limitations. First, they lack data analysis mechanisms for…

View free PDFSource page
arxivcs.LG2026-07-23

X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

Dongjie Fu, Di Cao, Xize Cheng, Zihan Zhang, Wenxu Jia, Yifu Chen, et al.

While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-O…

View free PDFSource page
arxiveess.SPcs.LG2026-07-23

Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models

Yingchao Huang, Xin Wang, Yuhan Su, Shanshan Yao

Cognitive impairment (CI) is a growing public health concern. Early and accurate diagnosis is critical for enabling timely intervention and improving patient outcomes. Speech-based CI detection has emerged as a promising non-invasive approach, as speech signals encode both lingui…

View free PDFSource page
arxivcs.ARcs.LG2026-07-23

DRC-Aid: Design-Rule Correction via Agentic Framework utilizing Inference-Time Large Language Models

Anushka Mukherjee, Kang He, Kaushik Roy

Resolving Design Rule Violations (DRVs) in layouts entails an iterative loop of geometric edits and verification. We present DRC-Aid, a closed-loop agentic framework that automates local DRC repair by formulating it as verification-in-the-loop search. To constrain the combinatori…

View free PDFSource page