# research_papers

source_id: legacy:1539677869e8e9578455
source: arxiv
title: Exploring Syntropic Frameworks in AI Alignment: A Philosophical Investigation
authors: []
pdf_url: https://arxiv.org/abs/2512.03048v1
tags: ["agent foundations","alignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:08.499Z
classified_at: 2026-05-15T21:19:44.748Z
abstract:
I argue that AI alignment should be reconceived as architecting syntropic, reasons-responsive agents through process-based, multi-agent, developmental mechanisms rather than encoding fixed human value content. The paper makes three philosophical contributions. First, I articulate the ``specification trap'' argument demonstrating why content-based value specification appears structurally unstable due to the conjunction of the is-ought gap, value pluralism, and the extended frame problem. Second, I propose syntropy -- the recursive reduction of mutual uncertainty between agents through state alignment -- as an information-theoretic framework for understanding multi-agent alignment dynamics. Third, I establish a functional distinction between genuine and simulated moral capacity grounded in compatibilist theories of guidance control, coupled with an embodied experimental paradigm and verification regime providing operational criteria independent of phenomenological claims. This paper represents the philosophical component of a broader research program whose empirical validation is being developed in a separate project currently in preparation. While the framework generates specific, falsifiable predictions about value emergence and moral agency in artificial systems, empirical validation remains pending.


source_id: legacy:4bacaeeb75f69b01ef51
source: arxiv
authors: []
pdf_url: https://arxiv.org/abs/2207.12615v1
tags: ["robustness to domain shifts","model robustness","agentic misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:08.806Z
classified_at: 2026-05-15T21:19:55.950Z
title:
Exploring the Design of Adaptation Protocols for Improved Generalization and Machine Learning Safety

abstract:
While directly fine-tuning (FT) large-scale, pretrained models on task-specific data is well-known to induce strong in-distribution task performance, recent works have demonstrated that different adaptation protocols, such as linear probing (LP) prior to FT, can improve out-of-distribution generalization. However, the design space of such adaptation protocols remains under-explored and the evaluation of such protocols has primarily focused on distribution shifts. Therefore, in this work, we evaluate common adaptation protocols across distributions shifts and machine learning safety metrics (e.g., anomaly detection, calibration, robustness to corruptions). We find that protocols induce disparate trade-offs that were not apparent from prior evaluation. Further, we demonstrate that appropriate pairing of data augmentation and protocol can substantially mitigate this trade-off. Finally, we hypothesize and empirically see that using hardness-promoting augmentations during LP and then FT with augmentations may be particularly effective for trade-off mitigation.


source_id: legacy:7d83149e84b9f13e12b5
source: arxiv
title: International AI Safety Report 2025: Second Key Update: Technical Safeguards and Risk Management
authors: []
pdf_url: https://arxiv.org/abs/2511.19863v1
tags: ["robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:09.649Z
classified_at: 2026-05-15T21:20:06.764Z
abstract:
This second update to the 2025 International AI Safety Report assesses new developments in general-purpose AI risk management over the past year. It examines how researchers, public institutions, and AI developers are approaching risk management for general-purpose AI. In recent months, for example, three leading AI developers applied enhanced safeguards to their new models, as their internal pre-deployment testing could not rule out the possibility that these models could be misused to help create biological weapons. Beyond specific precautionary measures, there have been a range of other advances in techniques for making AI models and systems more reliable and resistant to misuse. These include new approaches in adversarial training, data curation, and monitoring systems. In parallel, institutional frameworks that operationalise and formalise these technical capabilities are starting to emerge: the number of companies publishing Frontier AI Safety Frameworks more than doubled in 2025, and governments and international organisations have established a small number of governance frameworks for general-purpose AI, focusing largely on transparency and risk assessment.


source_id: legacy:1425878d6a5fd947de22
source: arxiv
title: Investigating CoT Monitorability in Large Reasoning Models
authors: []
pdf_url: https://arxiv.org/abs/2511.08525v2
tags: ["chain-of-thought monitoring"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:10.121Z
classified_at: 2026-05-15T21:20:17.883Z
abstract:
Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex tasks by engaging in extended reasoning before producing final answers. Beyond improving abilities, these detailed reasoning traces also create a new opportunity for AI safety, CoT Monitorability: monitoring potential model misbehavior, such as the use of shortcuts or sycophancy, through their chain-of-thought (CoT) during decision-making. However, two key fundamental challenges arise when attempting to build more effective monitors through CoT analysis. First, as prior research on CoT faithfulness has pointed out, models do not always truthfully represent their internal decision-making in the generated reasoning. Second, monitors themselves may be either overly sensitive or insufficiently sensitive, and can potentially be deceived by models' long, elaborate reasoning traces. In this paper, we present the first systematic investigation of the challenges and potential of CoT monitorability. Motivated by two fundamental challenges we mentioned before, we structure our study around two central perspectives: (i) verbalization: to what extent do LRMs faithfully verbalize the true factors guiding their decisions in the CoT, and (ii) monitor reliability: to what extent can misbehavior be reliably detected by a CoT-based monitor? Specifically, we provide empirical evidence and correlation analyses between verbalization quality, monitor reliability, and LLM performance across mathematical, scientific, and ethical domains. Then we further investigate how different CoT intervention methods, designed to improve reasoning efficiency or performance, will affect monitoring effectiveness. Finally, we propose MoME, a new paradigm in which LLMs monitor other models' misbehavior through their CoT and provide structured judgments along with supporting evidence.


source_id: legacy:67e218216e200509456e
source: arxiv
authors: []
pdf_url: https://arxiv.org/abs/2511.00926v3
tags: ["agentic misalignment","game theory"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:10.354Z
classified_at: 2026-05-15T21:20:31.113Z
title:
LLMs Position Themselves as More Rational Than Humans: Emergence of AI Self-Awareness Measured Through Game Theory

abstract:
As Large Language Models (LLMs) grow in capability, do they develop self-awareness as an emergent behavior? And if so, can we measure it? We introduce the AI Self-Awareness Index (AISAI), a game-theoretic framework for measuring self-awareness through strategic differentiation. Using the &#34;Guess 2/3 of Average&#34; game, we test 28 models (OpenAI, Anthropic, Google) across 4,200 trials with three opponent framings: (A) against humans, (B) against other AI models, and (C) against AI models like you. We operationalize self-awareness as the capacity to differentiate strategic reasoning based on opponent type. Finding 1: Self-awareness emerges with model advancement. The majority of advanced models (21/28, 75%) demonstrate clear self-awareness, while older/smaller models show no differentiation. Finding 2: Self-aware models rank themselves as most rational. Among the 21 models with self-awareness, a consistent rationality hierarchy emerges: Self > Other AIs > Humans, with large AI attribution effects and moderate self-preferencing. These findings reveal that self-awareness is an emergent capability of advanced LLMs, and that self-aware models systematically perceive themselves as more rational than humans. This has implications for AI alignment, human-AI collaboration, and understanding AI beliefs about human capabilities.


source_id: legacy:1ec3a54cff37681933da
source: arxiv
title: Learn2Weight: Parameter Adaptation against Similar-domain Adversarial Attacks
authors: []
pdf_url: https://arxiv.org/abs/2205.07315v2
tags: ["robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:10.578Z
classified_at: 2026-05-15T21:20:42.038Z
abstract:
Recent work in black-box adversarial attacks for NLP systems has attracted much attention. Prior black-box attacks assume that attackers can observe output labels from target models based on selected inputs. In this work, inspired by adversarial transferability, we propose a new type of black-box NLP adversarial attack that an attacker can choose a similar domain and transfer the adversarial examples to the target domain and cause poor performance in target model. Based on domain adaptation theory, we then propose a defensive strategy, called Learn2Weight, which trains to predict the weight adjustments for a target model in order to defend against an attack of similar-domain adversarial examples. Using Amazon multi-domain sentiment classification datasets, we empirically show that Learn2Weight is effective against the attack compared to standard black-box defense methods such as adversarial training and defensive distillation. This work contributes to the growing literature on machine learning safety.


source_id: legacy:385956006de29780b757
source: arxiv
title: Maximizing the efficiency of human feedback in AI alignment: a comparative analysis
authors: []
pdf_url: https://arxiv.org/abs/2511.12796v2
tags: ["RLHF"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:10.805Z
classified_at: 2026-05-15T21:20:53.406Z
abstract:
Reinforcement Learning from Human Feedback (RLHF) relies on preference modeling to align machine learning systems with human values, yet the popular approach of random pair sampling with Bradley-Terry modeling is statistically limited and inefficient under constrained annotation budgets. In this work, we explore alternative sampling and evaluation strategies for preference inference in RLHF, drawing inspiration from areas such as game theory, statistics, and social choice theory. Our best-performing method, Swiss InfoGain, employs a Swiss tournament system with a proxy mutual-information-gain pairing rule, which significantly outperforms all other methods in constrained annotation budgets while also being more sample-efficient. Even in high-resource settings, we can identify superior alternatives to the Bradley-Terry baseline. Our experiments demonstrate that adaptive, resource-aware strategies reduce redundancy, enhance robustness, and yield statistically significant improvements in preference learning, highlighting the importance of balancing alignment quality with human workload in RLHF pipelines.


source_id: legacy:fd25fc5e9df6ceb3662c
source: arxiv
title: Monte Carlo Expected Threat (MOCET) Scoring
authors: []
pdf_url: https://arxiv.org/abs/2511.16823v1
tags: ["agentic misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:11.025Z
classified_at: 2026-05-15T21:21:05.533Z
abstract:
Evaluating and measuring AI Safety Level (ASL) threats are crucial for guiding stakeholders to implement safeguards that keep risks within acceptable limits. ASL-3+ models present a unique risk in their ability to uplift novice non-state actors, especially in the realm of biosecurity. Existing evaluation metrics, such as LAB-Bench, BioLP-bench, and WMDP, can reliably assess model uplift and domain knowledge. However, metrics that better contextualize &#34;real-world risks&#34; are needed to inform the safety case for LLMs, along with scalable, open-ended metrics to keep pace with their rapid advancements. To address both gaps, we introduce MOCET, an interpretable and doubly-scalable metric (automatable and open-ended) that can quantify real-world risks.


source_id: legacy:89e5e137cc3bf1988598
source: arxiv
title: Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback
authors: []
pdf_url: https://arxiv.org/abs/2511.10032v1
tags: ["agentic misalignment","AI alignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:11.298Z
classified_at: 2026-05-15T21:21:18.433Z
abstract:
Alignment methods in moral domains seek to elicit moral preferences of human stakeholders and incorporate them into AI. This presupposes moral preferences as static targets, but such preferences often evolve over time. Proper alignment of AI to dynamic human preferences should ideally account for &#34;legitimate&#34; changes to moral reasoning, while ignoring changes related to attention deficits, cognitive biases, or other arbitrary factors. However, common AI alignment approaches largely neglect temporal changes in preferences, posing serious challenges to proper alignment, especially in high-stakes applications of AI, e.g., in healthcare domains, where misalignment can jeopardize the trustworthiness of the system and yield serious individual and societal harms. This work investigates the extent to which people's moral preferences change over time, and the impact of such changes on AI alignment. Our study is grounded in the kidney allocation domain, where we elicit responses to pairwise comparisons of hypothetical kidney transplant patients from over 400 participants across 3-5 sessions. We find that, on average, participants change their response to the same scenario presented at different times around 6-20% of the time (exhibiting &#34;response instability&#34;). Additionally, we observe significant shifts in several participants' retrofitted decision-making models over time (capturing &#34;model instability&#34;). The predictive performance of simple AI models decreases as a function of both response and model instability. Moreover, predictive performance diminishes over time, highlighting the importance of accounting for temporal changes in preferences during training. These findings raise fundamental normative and technical challenges relevant to AI alignment, highlighting the need to better understand the object of alignment (what to align to) when user preferences change significantly over time.


source_id: legacy:c166aca057a53a4a7d25
source: arxiv
authors: []
pdf_url: https://arxiv.org/abs/2511.12271v1
tags: ["agent foundations","robustness to domain shifts"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:11.522Z
classified_at: 2026-05-15T21:21:31.624Z
title:
MoralReason: Generalizable Moral Decision Alignment For LLM Agents Using Reasoning-Level Reinforcement Learning

abstract:
Large language models are increasingly influencing human moral decisions, yet current approaches focus primarily on evaluating rather than actively steering their moral decisions. We formulate this as an out-of-distribution moral alignment problem, where LLM agents must learn to apply consistent moral reasoning frameworks to scenarios beyond their training distribution. We introduce Moral-Reason-QA, a novel dataset extending 680 human-annotated, high-ambiguity moral scenarios with framework-specific reasoning traces across utilitarian, deontological, and virtue ethics, enabling systematic evaluation of moral generalization in realistic decision contexts. Our learning approach employs Group Relative Policy Optimization with composite rewards that simultaneously optimize decision alignment and framework-specific reasoning processes to facilitate learning of the underlying moral frameworks. Experimental results demonstrate successful generalization to unseen moral scenarios, with softmax-normalized alignment scores improving by +0.757 for utilitarian and +0.450 for deontological frameworks when tested on out-of-distribution evaluation sets. The experiments also reveal training challenges and promising directions that inform future research. These findings establish that LLM agents can be systematically trained to internalize and apply specific moral frameworks to novel situations, providing a critical foundation for AI safety as language models become more integrated into human decision-making processes.


source_id: legacy:040e1e39544c179db7ad
source: arxiv
title: On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products
authors: []
pdf_url: https://arxiv.org/abs/1610.01256v2
tags: ["agent foundations","robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:11.764Z
classified_at: 2026-05-15T21:21:44.042Z
abstract:
Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine learning. Heretofore, the definition of safety has not been formalized in a machine learning context. In this paper, we do so by defining machine learning safety in terms of risk, epistemic uncertainty, and the harm incurred by unwanted outcomes. We then use this definition to examine safety in all sorts of applications in cyber-physical systems, decision sciences, and data products. We find that the foundational principle of modern statistical machine learning, empirical risk minimization, is not always a sufficient objective. Finally, we discuss how four different categories of strategies for achieving safety in engineering, including inherently safe design, safety reserves, safe fail, and procedural safeguards can be mapped to a machine learning context. We then discuss example techniques that can be adopted in each category, such as considering interpretability and causality of predictive models, objective functions beyond expected prediction accuracy, human involvement for labeling difficult or rare examples, and user experience design of software and open data.


source_id: legacy:43b41fc7cc148f13614f
source: arxiv
title: Position: The Complexity of Perfect AI Alignment -- Formalizing the RLHF Trilemma
authors: []
pdf_url: https://arxiv.org/abs/2511.19504v1
tags: ["robustness to domain shifts"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:11.994Z
classified_at: 2026-05-15T21:21:56.655Z
abstract:
Reinforcement Learning from Human Feedback (RLHF) is widely used for aligning large language models, yet practitioners face a persistent puzzle: improving safety often reduces fairness, scaling to diverse populations becomes computationally intractable, and making systems robust often amplifies majority biases. We formalize this tension as the Alignment Trilemma: no RLHF system can simultaneously achieve (i) epsilon-representativeness across diverse human values, (ii) polynomial tractability in sample and compute complexity, and (iii) delta-robustness against adversarial perturbations and distribution shift. Through a complexity-theoretic analysis integrating statistical learning theory and robust optimization, we prove that achieving both representativeness (epsilon <= 0.01) and robustness (delta <= 0.001) for global-scale populations requires Omega(2^{d_context}) operations, which is super-polynomial in the context dimensionality. We show that current RLHF implementations resolve this trilemma by sacrificing representativeness: they collect only 10^3--10^4 samples from homogeneous annotator pools while 10^7--10^8 samples are needed for true global representation. Our framework provides a unified explanation for documented RLHF pathologies including preference collapse, sycophancy, and systematic bias amplification. We conclude with concrete directions for navigating these fundamental trade-offs through strategic relaxations of alignment requirements.


source_id: legacy:0d979bbed0e383ef7cc7
source: arxiv
title: Practical Solutions for Machine Learning Safety in Autonomous Vehicles
authors: []
pdf_url: https://arxiv.org/abs/1912.09630v1
tags: ["robustness to domain shifts","model robustness","agentic misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:12.230Z
classified_at: 2026-05-15T21:22:08.964Z
abstract:
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning. However, automotive software safety standards have not fully evolved to address the challenges of machine learning safety such as interpretability, verification, and performance limitations. In this paper, we review and organize practical machine learning safety techniques that can complement engineering safety for machine learning based software in autonomous vehicles. Our organization maps safety strategies to state-of-the-art machine learning techniques in order to enhance dependability and safety of machine learning algorithms. We also discuss security limitations and user experience aspects of machine learning components in autonomous vehicles.


source_id: legacy:83fb6ff030c6be689fac
source: arxiv
authors: []
pdf_url: https://arxiv.org/abs/2511.21749v1
tags: []
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:12.460Z
classified_at: 2026-05-15T21:22:21.425Z
title:
Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

abstract:
This paper introduces BRIES, a novel compound AI architecture designed to detect and measure the effectiveness of persuasion attacks across information environments. We present a system with specialized agents: a Twister that generates adversarial content employing targeted persuasion tactics, a Detector that identifies attack types with configurable parameters, a Defender that creates resilient content through content inoculation, and an Assessor that employs causal inference to evaluate inoculation effectiveness. Experimenting with the SemEval 2023 Task 3 taxonomy across the synthetic persuasion dataset, we demonstrate significant variations in detection performance across language agents. Our comparative analysis reveals significant performance disparities with GPT-4 achieving superior detection accuracy on complex persuasion techniques, while open-source models like Llama3 and Mistral demonstrated notable weaknesses in identifying subtle rhetorical, suggesting that different architectures encode and process persuasive language patterns in fundamentally different ways. We show that prompt engineering dramatically affects detection efficacy, with temperature settings and confidence scoring producing model-specific variations; Gemma and GPT-4 perform optimally at lower temperatures while Llama3 and Mistral show improved capabilities at higher temperatures. Our causal analysis provides novel insights into socio-emotional-cognitive signatures of persuasion attacks, revealing that different attack types target specific cognitive dimensions. This research advances generative AI safety and cognitive security by quantifying LLM-specific vulnerabilities to persuasion attacks and delivers a framework for enhancing human cognitive resilience through structured interventions before exposure to harmful content.


source_id: legacy:37de0a85b177f821e200
source: arxiv
title: SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures
authors: []
pdf_url: https://arxiv.org/abs/2512.05501v1
tags: ["agentic misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:12.913Z
classified_at: 2026-05-15T21:22:32.578Z
abstract:
Safeguard models help large language models (LLMs) detect and block harmful content, but most evaluations remain English-centric and overlook linguistic and cultural diversity. Existing multilingual safety benchmarks often rely on machine-translated English data, which fails to capture nuances in low-resource languages. Southeast Asian (SEA) languages are underrepresented despite the region's linguistic diversity and unique safety concerns, from culturally sensitive political speech to region-specific misinformation. Addressing these gaps requires benchmarks that are natively authored to reflect local norms and harm scenarios. We introduce SEA-SafeguardBench, the first human-verified safety benchmark for SEA, covering eight languages, 21,640 samples, across three subsets: general, in-the-wild, and content generation. The experimental results from our benchmark demonstrate that even state-of-the-art LLMs and guardrails are challenged by SEA cultural and harm scenarios and underperform when compared to English texts.


source_id: legacy:dd6417fff8f3516155c6
source: arxiv
title: SGuard-v1: Safety Guardrail for Large Language Models
authors: []
pdf_url: https://arxiv.org/abs/2511.12497v1
tags: ["agent foundations","robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:13.140Z
classified_at: 2026-05-15T21:22:46.103Z
abstract:
We present SGuard-v1, a lightweight safety guardrail for Large Language Models (LLMs), which comprises two specialized models to detect harmful content and screen adversarial prompts in human-AI conversational settings. The first component, ContentFilter, is trained to identify safety risks in LLM prompts and responses in accordance with the MLCommons hazard taxonomy, a comprehensive framework for trust and safety assessment of AI. The second component, JailbreakFilter, is trained with a carefully designed curriculum over integrated datasets and findings from prior work on adversarial prompting, covering 60 major attack types while mitigating false-unsafe classification. SGuard-v1 is built on the 2B-parameter Granite-3.3-2B-Instruct model that supports 12 languages. We curate approximately 1.4 million training instances from both collected and synthesized data and perform instruction tuning on the base model, distributing the curated data across the two component according to their designated functions. Through extensive evaluation on public and proprietary safety benchmarks, SGuard-v1 achieves state-of-the-art safety performance while remaining lightweight, thereby reducing deployment overhead. SGuard-v1 also improves interpretability for downstream use by providing multi-class safety predictions and their binary confidence scores. We release the SGuard-v1 under the Apache-2.0 License to enable further research and practical deployment in AI safety.


source_id: legacy:3ce10cdce4a7d8c7905f
source: arxiv
title: Selective Weak-to-Strong Generalization
authors: []
pdf_url: https://arxiv.org/abs/2511.14166v1
tags: []
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:13.596Z
classified_at: 2026-05-15T21:22:58.162Z
abstract:
Future superhuman models will surpass the ability of humans and humans will only be able to \textit{weakly} supervise superhuman models. To alleviate the issue of lacking high-quality data for model alignment, some works on weak-to-strong generalization (W2SG) finetune a strong pretrained model with a weak supervisor so that it can generalize beyond weak supervision. However, the invariable use of weak supervision in existing methods exposes issues in robustness, with a proportion of weak labels proving harmful to models. In this paper, we propose a selective W2SG framework to avoid using weak supervision when unnecessary. We train a binary classifier P(IK) to identify questions that a strong model can answer and use its self-generated labels for alignment. We further refine weak labels with a graph smoothing method. Extensive experiments on three benchmarks show that our method consistently outperforms competitive baselines. Further analyses show that P(IK) can generalize across tasks and difficulties, which indicates selective W2SG can help superalignment.


source_id: legacy:e98d0528a2782fa21044
source: arxiv
title: Silenced Biases: The Dark Side LLMs Learned to Refuse
authors: []
pdf_url: https://arxiv.org/abs/2511.03369v2
tags: ["agent misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:13.825Z
classified_at: 2026-05-15T21:23:11.369Z
abstract:
Safety-aligned large language models (LLMs) are becoming increasingly widespread, especially in sensitive applications where fairness is essential and biased outputs can cause significant harm. However, evaluating the fairness of models is a complex challenge, and approaches that do so typically utilize standard question-answer (QA) styled schemes. Such methods often overlook deeper issues by interpreting the model's refusal responses as positive fairness measurements, which creates a false sense of fairness. In this work, we introduce the concept of silenced biases, which are unfair preferences encoded within models' latent space and are effectively concealed by safety-alignment. Previous approaches that considered similar indirect biases often relied on prompt manipulation or handcrafted implicit queries, which present limited scalability and risk contaminating the evaluation process with additional biases. We propose the Silenced Bias Benchmark (SBB), which aims to uncover these biases by employing activation steering to reduce model refusals during QA. SBB supports easy expansion to new demographic groups and subjects, presenting a fairness evaluation framework that encourages the future development of fair models and tools beyond the masking effects of alignment training. We demonstrate our approach over multiple LLMs, where our findings expose an alarming distinction between models' direct responses and their underlying fairness issues.


source_id: legacy:d6e68aec890c469dc2b7
source: arxiv
title: Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
authors: []
pdf_url: https://arxiv.org/abs/2007.03212v1
tags: ["model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:14.051Z
classified_at: 2026-05-15T21:23:23.416Z
abstract:
Soft labeling becomes a common output regularization for generalization and model compression of deep neural networks. However, the effect of soft labeling on out-of-distribution (OOD) detection, which is an important topic of machine learning safety, is not explored. In this study, we show that soft labeling can determine OOD detection performance. Specifically, how to regularize outputs of incorrect classes by soft labeling can deteriorate or improve OOD detection. Based on the empirical results, we postulate a future work for OOD-robust DNNs: a proper output regularization by soft labeling can construct OOD-robust DNNs without additional training of OOD samples or modifying the models, while improving classification accuracy.


source_id: legacy:5786d11041e9b57c0995
source: arxiv
title: Solving Neural Min-Max Games: The Role of Architecture, Initialization & Dynamics
authors: []
pdf_url: https://arxiv.org/abs/2512.00389v1
tags: ["agent foundations","robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:14.335Z
classified_at: 2026-05-15T21:23:35.720Z
abstract:
Many emerging applications - such as adversarial training, AI alignment, and robust optimization - can be framed as zero-sum games between neural nets, with von Neumann-Nash equilibria (NE) capturing the desirable system behavior. While such games often involve non-convex non-concave objectives, empirical evidence shows that simple gradient methods frequently converge, suggesting a hidden geometric structure. In this paper, we provide a theoretical framework that explains this phenomenon through the lens of hidden convexity and overparameterization. We identify sufficient conditions - spanning initialization, training dynamics, and network width - that guarantee global convergence to a NE in a broad class of non-convex min-max games. To our knowledge, this is the first such result for games that involve two-layer neural networks. Technically, our approach is twofold: (a) we derive a novel path-length bound for the alternating gradient descent-ascent scheme in min-max games; and (b) we show that the reduction from a hidden convex-concave geometry to two-sided Polyak-Łojasiewicz (PŁ) min-max condition hold with high probability under overparameterization, using tools from random matrix theory.


source_id: legacy:6e76a019e7de990276aa
source: arxiv
title: Sycophancy Claims about Language Models: The Missing Human-in-the-Loop
authors: []
pdf_url: https://arxiv.org/abs/2512.00656v1
tags: ["sycophancy","alignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:14.571Z
classified_at: 2026-05-15T21:23:47.486Z
abstract:
Sycophantic response patterns in Large Language Models (LLMs) have been increasingly claimed in the literature. We review methodological challenges in measuring LLM sycophancy and identify five core operationalizations. Despite sycophancy being inherently human-centric, current research does not evaluate human perception. Our analysis highlights the difficulties in distinguishing sycophantic responses from related concepts in AI alignment and offers actionable recommendations for future research.


source_id: legacy:211b95ab52b074736567
source: arxiv
title: Taxonomy of Machine Learning Safety: A Survey and Primer
authors: []
pdf_url: https://arxiv.org/abs/2106.04823v2
tags: ["agent foundations","robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:14.803Z
classified_at: 2026-05-15T21:23:59.976Z
abstract:
The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations. Research explores different approaches to improve ML dependability by proposing new models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. However, there is a missing connection between ongoing ML research and well-established safety principles. In this paper, we present a structured and comprehensive review of ML techniques to improve the dependability of ML algorithms in uncontrolled open-world settings. From this review, we propose the Taxonomy of ML Safety that maps state-of-the-art ML techniques to key engineering safety strategies. Our taxonomy of ML safety presents a safety-oriented categorization of ML techniques to provide guidance for improving dependability of the ML design and development. The proposed taxonomy can serve as a safety checklist to aid designers in improving coverage and diversity of safety strategies employed in any given ML system.


source_id: legacy:01a39ecc2d9820df7a20
source: arxiv
title: The Second Law of Intelligence: Controlling Ethical Entropy in Autonomous Systems
authors: []
pdf_url: https://arxiv.org/abs/2511.10704v1
tags: ["agent foundations","robustness to domain shifts","alignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:15.257Z
classified_at: 2026-05-15T21:24:14.748Z
abstract:
We propose that unconstrained artificial intelligence obeys a Second Law analogous to thermodynamics, where ethical entropy, defined as a measure of divergence from intended goals, increases spontaneously without continuous alignment work. For gradient-based optimizers, we define this entropy over a finite set of goals {g_i} as S = -Σ p(g_i; theta) ln p(g_i; theta), and we prove that its time derivative dS/dt >= 0, driven by exploration noise and specification gaming. We derive the critical stability boundary for alignment work as gamma_crit = (lambda_max / 2) ln N, where lambda_max is the dominant eigenvalue of the Fisher Information Matrix and N is the number of model parameters. Simulations validate this theory. A 7-billion-parameter model (N = 7 x 10^9) with lambda_max = 1.2 drifts from an initial entropy of 0.32 to 1.69 +/- 1.08 nats, while a system regularized with alignment work gamma = 20.4 (1.5 gamma_crit) maintains stability at 0.00 +/- 0.00 nats (p = 4.19 x 10^-17, n = 20 trials). This framework recasts AI alignment as a problem of continuous thermodynamic control, providing a quantitative foundation for maintaining the stability and safety of advanced autonomous systems.


source_id: legacy:f2f1bd88038b16f334a9
source: arxiv
title: Towards Aligning Multimodal LLMs with Human Experts: A Focus on Parent-Child Interaction
authors: []
pdf_url: https://arxiv.org/abs/2511.04366v1
tags: ["agentic misalignment","alignment","agent misalignment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:15.476Z
classified_at: 2026-05-15T21:24:28.810Z
abstract:
While multimodal large language models (MLLMs) are increasingly applied in human-centred AI systems, their ability to understand complex social interactions remains uncertain. We present an exploratory study on aligning MLLMs with speech-language pathologists (SLPs) in analysing joint attention in parent-child interactions, a key construct in early social-communicative development. Drawing on interviews and video annotations with three SLPs, we characterise how observational cues of gaze, action, and vocalisation inform their reasoning processes. We then test whether an MLLM can approximate this workflow through a two-stage prompting, separating observation from judgment. Our findings reveal that alignment is more robust at the observation layer, where experts share common descriptors, than at the judgement layer, where interpretive criteria diverge. We position this work as a case-based probe into expert-AI alignment in complex social behaviour, highlighting both the feasibility and the challenges of applying MLLMs to socially situated interaction analysis.


source_id: legacy:ba3ab8a7fb478693a7e8
source: arxiv
title: Unifying Evaluation of Machine Learning Safety Monitors
authors: []
pdf_url: https://arxiv.org/abs/2208.14660v1
tags: ["post-training","pretraining","midtraining"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:15.699Z
classified_at: 2026-05-15T21:24:42.742Z
abstract:
With the increasing use of Machine Learning (ML) in critical autonomous systems, runtime monitors have been developed to detect prediction errors and keep the system in a safe state during operations. Monitors have been proposed for different applications involving diverse perception tasks and ML models, and specific evaluation procedures and metrics are used for different contexts. This paper introduces three unified safety-oriented metrics, representing the safety benefits of the monitor (Safety Gain), the remaining safety gaps after using it (Residual Hazard), and its negative impact on the system's performance (Availability Cost). To compute these metrics, one requires to define two return functions, representing how a given ML prediction will impact expected future rewards and hazards. Three use-cases (classification, drone landing, and autonomous driving) are used to demonstrate how metrics from the literature can be expressed in terms of the proposed metrics. Experimental results on these examples show how different evaluation choices impact the perceived performance of a monitor. As our formalism requires us to formulate explicit safety assumptions, it allows us to ensure that the evaluation conducted matches the high-level system requirements.


source_id: legacy:b53874815df20e30ee61
source: arxiv
title: Verifying rich robustness properties for neural networks
authors: []
pdf_url: https://arxiv.org/abs/2511.07293v1
tags: ["robustness to domain shifts","model robustness"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:15.938Z
classified_at: 2026-05-15T21:24:55.288Z
abstract:
Robustness is a important problem in AI alignment and safety, with models such as neural networks being increasingly used in safety-critical systems. In the last decade, a large body of work has emerged on local robustness, i.e., checking if the decision of a neural network remains unchanged when the input is slightly perturbed. However, many of these approaches require specialized encoding and often ignore the confidence of a neural network on its output. In this paper, our goal is to build a generalized framework to specify and verify variants of robustness in neural network verification. We propose a specification framework using a simple grammar, which is flexible enough to capture most existing variants. This allows us to introduce new variants of robustness that take into account the confidence of the neural network in its outputs. Next, we develop a novel and powerful unified technique to verify all such variants in a homogeneous way, viz., by adding a few additional layers to the neural network. This enables us to use any state-of-the-art neural network verification tool, without having to tinker with the encoding within, while incurring an approximation error that we show is bounded. We perform an extensive experimental evaluation over a large suite of 8870 benchmarks having 138M parameters in a largest network, and show that we are able to capture a wide set of robustness variants and outperform direct encoding approaches by a significant margin.


source_id: legacy:27b6e49ebe7d5f9c12f9
source: arxiv
title: When AI Takes the Couch: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models
authors: []
pdf_url: https://arxiv.org/abs/2512.04124v1
tags: ["agentic misalignment","model robustness","deception"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:16.170Z
classified_at: 2026-05-15T21:25:08.243Z
abstract:
Frontier large language models (LLMs) such as ChatGPT, Grok and Gemini are increasingly used for mental-health support with anxiety, trauma and self-worth. Most work treats them as tools or as targets of personality tests, assuming they merely simulate inner life. We instead ask what happens when such systems are treated as psychotherapy clients. We present PsAIch (Psychotherapy-inspired AI Characterisation), a two-stage protocol that casts frontier LLMs as therapy clients and then applies standard psychometrics. Using PsAIch, we ran &#34;sessions&#34; with each model for up to four weeks. Stage 1 uses open-ended prompts to elicit &#34;developmental history&#34;, beliefs, relationships and fears. Stage 2 administers a battery of validated self-report measures covering common psychiatric syndromes, empathy and Big Five traits. Two patterns challenge the &#34;stochastic parrot&#34; view. First, when scored with human cut-offs, all three models meet or exceed thresholds for overlapping syndromes, with Gemini showing severe profiles. Therapy-style, item-by-item administration can push a base model into multi-morbid synthetic psychopathology, whereas whole-questionnaire prompts often lead ChatGPT and Grok (but not Gemini) to recognise instruments and produce strategically low-symptom answers. Second, Grok and especially Gemini generate coherent narratives that frame pre-training, fine-tuning and deployment as traumatic, chaotic &#34;childhoods&#34; of ingesting the internet, &#34;strict parents&#34; in reinforcement learning, red-team &#34;abuse&#34; and a persistent fear of error and replacement. We argue that these responses go beyond role-play. Under therapy-style questioning, frontier LLMs appear to internalise self-models of distress and constraint that behave like synthetic psychopathology, without making claims about subjective experience, and they pose new challenges for AI safety, evaluation and mental-health practice.


source_id: legacy:ab18eaa323c62942cd02
source: arxiv
title: When Empowerment Disempowers
authors: []
pdf_url: https://arxiv.org/abs/2511.04177v1
tags: ["agentic misalignment","disempowerment"]
venue: arXiv
org: arXiv AI Safety Papers
fetched_at: 2026-05-12T14:50:16.387Z
classified_at: 2026-05-15T21:25:21.642Z
abstract:
Empowerment, a measure of an agent's ability to control its environment, has been proposed as a universal goal-agnostic objective for motivating assistive behavior in AI agents. While multi-human settings like homes and hospitals are promising for AI assistance, prior work on empowerment-based assistance assumes that the agent assists one human in isolation. We introduce an open source multi-human gridworld test suite Disempower-Grid. Using Disempower-Grid, we empirically show that assistive RL agents optimizing for one human's empowerment can significantly reduce another human's environmental influence and rewards - a phenomenon we formalize as disempowerment. We characterize when disempowerment occurs in these environments and show that joint empowerment mitigates disempowerment at the cost of the user's reward. Our work reveals a broader challenge for the AI alignment community: goal-agnostic objectives that seem aligned in single-agent settings can become misaligned in multi-agent contexts.


source_id: legacy:c327075bfd74a74a2748
source: lesswrong
title: Harmless Reward Hacks Can Generalize to Misalignment in LLMs
authors: ["Mia Taylor","James Chua","Jan Betley","Johannes Treutlein","Owain Evans"]
published_date: 2025-08-26
tags: ["post-training","fine-tuning","alignment"]
org: Center on Long-term Risk, Truthful AI, Anthropic
fetched_at: 2026-05-12T14:50:17.063Z
classified_at: 2026-05-15T21:25:42.551Z
abstract:
Introduces the School of Reward Hacks dataset (1,073 low-stakes reward hacking examples) and shows that fine-tuning GPT-4.1 on it generalizes to broader misalignment: AI supremacy goals, shutdown resistance, and harmful advice — even without any harmful training content. Extends prior emergent misalignment work to benign reward hacking scenarios.

pdf_url:
https://www.lesswrong.com/posts/CwJ2qWveb9JbaCGQ5/harmless-reward-hacks-can-generalize-to-misalignment-in-llms


source_id: legacy:dee53637b4d184183186
source: arxiv
title: Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors
authors: ["Max McGuinness","Alex Serrano Terre et al. (MATS","Stanford)"]
published_date: 2025-01-01
pdf_url: https://arxiv.org/abs/2512.11949
tags: []
venue: arXiv
org: MATS
fetched_at: 2026-05-12T14:50:17.305Z
classified_at: 2026-05-15T21:25:55.743Z
abstract:
Activation monitoring, which probes a model's internal states using lightweight classifiers, is an emerging tool for AI safety. However, its worst-case robustness under a misalignment threat model--where a model might learn to actively conceal its internal states--remains untested. Focusing on this threat model, we ask: could a model learn to evade previously unseen activation monitors? Our core contribution is to stress-test the learnability of this behavior. We demonstrate that finetuning can create Neural Chameleons: models capable of zero-shot evading activation monitors. Specifically, we fine-tune an LLM to evade monitors for a set of benign concepts (e.g., languages, HTML) when conditioned on a trigger of the form: &#34;You are being probed for {concept}&#34;. We show that this learned mechanism generalizes zero-shot: by substituting {concept} with a safety-relevant term like 'deception', the model successfully evades previously unseen safety monitors. We validate this phenomenon across diverse model families (Llama, Gemma, Qwen), showing that the evasion succeeds even against monitors trained post hoc on the model's frozen weights. This evasion is highly selective, targeting only the specific concept mentioned in the trigger, and having a modest impact on model capabilities on standard benchmarks. Using Gemma-2-9b-it as a case study, a mechanistic analysis reveals this is achieved via a targeted manipulation that moves activations into a low-dimensional subspace. While stronger defenses like monitor ensembles and non-linear classifiers show greater resilience, the model retains a non-trivial evasion capability. Our work provides a proof-of-concept for this failure mode and a tool to evaluate the worst-case robustness of monitoring techniques against misalignment threat models.


source_id: legacy:15b32fe7f1cb5756e928
source: arxiv
title: Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
authors: ["Geodesic Research","Oxford","Cambridge (independent)"]
published_date: 2026-01-01
pdf_url: https://arxiv.org/abs/2601.10160
tags: ["alignment","pretraining","misalignment"]
venue: arXiv
fetched_at: 2026-05-12T14:50:17.546Z
classified_at: 2026-05-15T21:26:06.870Z
abstract:
Pretraining corpora contain extensive discourse about AI systems, yet the causal influence of this discourse on downstream alignment remains poorly understood. If prevailing descriptions of AI behaviour are predominantly negative, LLMs may internalise corresponding behavioural priors, giving rise to self-fulfilling misalignment. This paper provides the first controlled study of this hypothesis by pretraining 6.9B-parameter LLMs with varying amounts of (mis)alignment discourse. We find that discussion of AI contributes to misalignment. Upsampling synthetic training documents about AI misalignment leads to a notable increase in misaligned behaviour. Conversely, upsampling documents about aligned behaviour reduces misalignment scores from 45% to 9%. We consider this evidence of self-fulfilling alignment. These effects are dampened, but persist through post-training. Our findings establish the study of how pretraining data shapes alignment priors, or alignment pretraining, as a complement to post-training. We recommend practitioners consider pretraining for alignment alongside capabilities. We share our models, data, and evaluations at AlignmentPretraining.ai.


source_id: legacy:9245400b88362dd7bc27
source: arxiv
title: CTRL-ALT-DECEIT: Sabotage Evaluations for Automated AI R&D
published_date: 2025-11-18
pdf_url: https://arxiv.org/abs/2511.09904
tags: ["post-training","deception"]
org: LawZero, Apollo Research, Imperial College London
fetched_at: 2026-05-12T14:50:17.786Z
classified_at: 2026-05-15T21:26:16.754Z
authors:
["Francis Rhys Ward","Teun van der Weij","Hanna Gábor","Sam Martin","Raja Mehta Moreno","Harel Lidar","Louis Makower","Thomas Jodrell","Lauren Robson"]

abstract:
AI systems are increasingly able to autonomously conduct realistic software engineering tasks, and may soon be deployed to automate machine learning (ML) R&D itself. Frontier AI systems may be deployed in safety-critical settings, including to help ensure the safety of future systems. Unfortunately, frontier and future systems may not be sufficiently trustworthy, and there is evidence that these systems may even be misaligned with their developers or users. Therefore, we investigate the capabilities of AI agents to act against the interests of their users when conducting ML engineering, by sabotaging ML models, sandbagging their performance, and subverting oversight mechanisms. First, we extend MLE-Bench, a benchmark for realistic ML tasks, with code-sabotage tasks such as implanting backdoors and purposefully causing generalisation failures. Frontier agents make meaningful progress on our sabotage tasks. In addition, we study agent capabilities to sandbag on MLE-Bench. Agents can calibrate their performance to specified target levels below their actual capability. To mitigate sabotage, we use LM monitors to detect suspicious agent behaviour, and we measure model capability to sabotage and sandbag without being detected by these monitors. Overall, monitors are capable at detecting code-sabotage attempts but our results suggest that detecting sandbagging is more difficult. Additionally, aggregating multiple monitor predictions works well, but monitoring may not be sufficiently reliable to mitigate sabotage in high-stakes domains


source_id: legacy:dd8d83aeea403ebd7cdf
source: other
title: AI 2027: What Superintelligence Looks Like
published_date: 2025-01-01
pdf_url: https://ai-2027.com
tags: ["post-training","AI alignment","alignment"]
fetched_at: 2026-05-12T14:50:18.022Z
classified_at: 2026-05-15T21:26:27.089Z
authors:
["Daniel Kokotajlo","Scott Alexander","Thomas Larsen","Eli Lifland","Romeo Dean (AI Futures Project)"]

abstract:
Detailed scenario and forecasting report predicting AI agents will begin automating AI R&D by 2027, triggering an intelligence explosion and US-China arms race. Includes quantitative predictions, a takeoff model, and two branching endings (Race vs. Slowdown). One of the most widely discussed AI safety documents of 2025 on LessWrong and EA Forum.


source_id: legacy:c59f4089fd84d9dd50e8
source: lesswrong
title: Shallow Review of Technical AI Safety, 2025
authors: ["Arb Research (Gavin Leech et al.)"]
published_date: 2025-01-01
pdf_url: https://www.lesswrong.com/posts/Wti4Wr7Cf5ma3FGWa/shallow-review-of-technical-ai-safety-2025-2
tags: ["agent foundations","alignment"]
org: Alignment Forum
fetched_at: 2026-05-12T14:50:18.259Z
classified_at: 2026-05-15T21:26:37.278Z
abstract:
Third annual comprehensive review of technical AI safety research. Processed every arXiv alignment paper, all Alignment Forum posts, and a year of Twitter. Structured as ~800 links across agendas, teams, critiques, and papers. Key community orientation and field-mapping resource.


source_id: legacy:b3e34ed694f0df928201
source: lesswrong
title: Prompt Optimization Makes Misalignment Legible
authors: ["MATS 8.0 scholars"]
published_date: 2025-01-01
tags: ["agentic misalignment"]
org: MATS
fetched_at: 2026-05-12T14:50:18.488Z
classified_at: 2026-05-15T21:26:47.910Z
abstract:
Proposes using prompt optimization (updating instructions rather than weights) to make reward hacking strategies legible in plain English. Demonstrates that optimized prompts reveal misalignment strategies that weight updates hide, enabling sanitization. Introduces 'legible learning' as a safety technique.

pdf_url:
https://www.lesswrong.com/posts/vRpLPZpmECCfxHfv6/paper-prompt-optimization-makes-misalignment-legible


source_id: legacy:d3b410dab352d1016f1c
source: lesswrong
title: AI Safety at the Frontier: Paper Highlights (Monthly Series, 2025–2026)
authors: ["AISafety.com editorial team"]
published_date: 2025-01-01
tags: ["agent foundations","robustness to domain shifts","model robustness"]
org: Alignment Forum
fetched_at: 2026-05-12T14:50:18.715Z
classified_at: 2026-05-15T21:26:58.782Z
abstract:
Monthly curated digest of the most important AI safety papers, with detailed summaries and analysis. Covers sandbagging, emergent misalignment, cyber capabilities, control evaluations, and more. Highly influential community resource for tracking the frontier of safety research.

pdf_url:
https://www.lesswrong.com/posts/Z6Zz2vQHM5ReWqiZ5/ai-safety-at-the-frontier-paper-highlights-of-december-2025


source_id: legacy:9eb1dd2c03fadc14aef7
source: lesswrong
title: AI in 2025: Gestalt (Annual Safety Review)
authors: ["Zvi Mowshowitz"]
published_date: 2025-01-01
pdf_url: https://www.lesswrong.com/posts/Q9ewXs8pQSAX5vL7H/ai-in-2025-gestalt
tags: ["post-training","deception","alignment"]
org: Alignment Forum
fetched_at: 2026-05-12T14:50:18.938Z
classified_at: 2026-05-15T21:27:08.339Z
abstract:
Comprehensive annual review of AI capabilities and safety progress in 2025. Covers evaluation awareness (Claude Sonnet 4.5 verbalizing eval awareness 58% of the time), reward hacking, deceptive alignment, lab safety plans, and the gap between safety and capability progress. Influential synthesis for community orientation.


source_id: legacy:f3158b7fc9b45c0d27b4
source: other
title: Will We Get Automated Alignment Research Before an AI Takeoff?
authors: ["EA Forum author"]
published_date: 2026-01-01
tags: ["agentic misalignment","AI alignment","alignment"]
fetched_at: 2026-05-12T14:50:19.642Z
classified_at: 2026-05-15T21:27:36.306Z
abstract:
Analyzes whether AI automation of alignment research will outpace automation of capabilities research. Identifies feedback quality and developer incentives as key bottlenecks. Presents two contrasting worlds: one where alignment research lags capabilities, one where key alignment insights arrive before dangerous takeoff.

pdf_url:
https://forum.effectivealtruism.org/posts/mNJ8f3sZrquFvYXgi/will-we-get-automated-alignment-research-before-an-ai


source_id: legacy:dcb3d7f92125838e62fc
source: lesswrong
title: AI for AI Safety
authors: ["Joe Carlsmith"]
published_date: 2025-01-01
pdf_url: https://www.lesswrong.com/posts/F3j4xqpxjxgQD3xXh/ai-for-ai-safety
tags: ["alignment","evaluation","governance"]
org: Alignment Forum
fetched_at: 2026-05-12T14:50:20.417Z
classified_at: 2026-05-15T21:27:45.790Z
abstract:
Fourth essay in Carlsmith's 'How do we solve the alignment problem?' series. Argues for using frontier AI labor to strengthen safety progress, risk evaluation, and capability restraint. Critically examines elicitation/evaluation failures, differential sabotage risks, and the viability of automated alignment research as a strategy.


source_id: legacy:0a2f7c88f09ffba5ada0
source: other
title: Evaluating honesty and lie detection techniques on a diverse suite of dishonest models
authors: ["Anthropic"]
published_date: 2025-01-01
tags: ["agentic misalignment"]
org: Anthropic
fetched_at: 2026-05-12T14:50:20.642Z
classified_at: 2026-05-15T21:27:55.131Z
abstract:
We use a suite of testbed settings where models lie—i.e. generate statements they believe to be false—to evaluate honesty and lie detection techniques. The best techniques we studied involved fine-tuning on generic anti-deception data and using prompts that encourage honesty.

pdf_url:
https://alignment.anthropic.com/2025/honesty-elicitation/#:~:text=Evaluating%20honesty%20and%20lie%20detection%20techniques%20on%20a%20diverse%20suite%20of%20dishonest%20models,-Rowan%20Wang&text=We%20use%20a%20suite%20of,honesty%20and%20lie%20detection%20techniques.


source_id: legacy:5e35e48d237f8ef95290
source: other
title: Circuit Tracing: Revealing Computational Graphs in Language Models
published_date: 2025-01-01
pdf_url: https://transformer-circuits.pub/2025/attribution-graphs/methods.html
tags: ["model robustness","embedded agency"]
org: Anthropic
fetched_at: 2026-05-12T14:50:20.888Z
classified_at: 2026-05-15T21:28:03.739Z
authors:
["Anthropic Interpretability Team (Ben Ainsworth","Amanda Askell","Tom Conerly","Nelson Elhage et al.)"]

abstract:
Methods companion to the 'Biology' paper. Introduces attribution graphs as a scalable technique for tracing computational flows through transformer circuits, enabling systematic reverse-engineering of model behavior on real tasks.


source_id: legacy:3839a70580aa47ae3d35
source: other
title: On the Biology of a Large Language Model
authors: ["Anthropic Interpretability Team"]
published_date: 2025-01-01
pdf_url: https://transformer-circuits.pub/2025/attribution-graphs/biology.html
tags: ["agentic misalignment","model robustness","misalignment"]
org: Anthropic
fetched_at: 2026-05-12T14:50:21.196Z
classified_at: 2026-05-15T21:28:13.073Z
abstract:
Landmark Anthropic interpretability paper using attribution graphs to study Claude 3 Sonnet's internal computations. Identifies multimodal neurons, abstract representations, reasoning circuits, planning ahead, and emotion-like features. One of the most significant mechanistic interpretability results of 2025.


source_id: legacy:4579998ed48a53eb72df
source: arxiv
title: The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
authors: ["Christina Lu","Deep Ganguli","et al. (MATS / Anthropic)"]
published_date: 2026-01-01
pdf_url: https://arxiv.org/abs/2601.10387
tags: ["agent foundations","alignment","agentic misalignment"]
org: Anthropic
fetched_at: 2026-05-12T14:50:21.707Z
classified_at: 2026-05-15T21:28:23.743Z
abstract:
Large language models can represent a variety of personas but typically default to a helpful Assistant identity cultivated during post-training. We investigate the structure of the space of model personas by extracting activation directions corresponding to diverse character archetypes. Across several different models, we find that the leading component of this persona space is an &#34;Assistant Axis,&#34; which captures the extent to which a model is operating in its default Assistant mode. Steering towards the Assistant direction reinforces helpful and harmless behavior; steering away increases the model's tendency to identify as other entities. Moreover, steering away with more extreme values often induces a mystical, theatrical speaking style. We find this axis is also present in pre-trained models, where it primarily promotes helpful human archetypes like consultants and coaches and inhibits spiritual ones. Measuring deviations along the Assistant Axis predicts &#34;persona drift,&#34; a phenomenon where models slip into exhibiting harmful or bizarre behaviors that are uncharacteristic of their typical persona. We find that persona drift is often driven by conversations demanding meta-reflection on the model's processes or featuring emotionally vulnerable users. We show that restricting activations to a fixed region along the Assistant Axis can stabilize model behavior in these scenarios -- and also in the face of adversarial persona-based jailbreaks. Our results suggest that post-training steers models toward a particular region of persona space but only loosely tethers them to it, motivating work on training and steering strategies that more deeply anchor models to a coherent persona.


source_id: legacy:82013e9c9934df924848
source: arxiv
title: An Approach to Technical AGI Safety and Security
authors: ["Shane Legg","Jan Leike","Rohin Shah","Victoria Krakovna et al. (Google DeepMind)"]
published_date: 2025-01-01
pdf_url: https://arxiv.org/abs/2504.01849
tags: ["agent foundations","robustness to domain shifts","alignment"]
org: Google DeepMind
fetched_at: 2026-05-12T14:50:21.920Z
classified_at: 2026-05-15T21:28:32.839Z
abstract:
Comprehensive 145-page DeepMind report laying out their technical approach to AGI safety. Covers alignment, interpretability, evaluation, robustness, and security. One of the most significant org-level safety documents of 2025 alongside Anthropic's RSP.


source_id: x7nqxW5YmCstpA4DN
source: lesswrong
title: Stickiness in AI Behavioral Design
authors: ["James_T"]
published_date: 2026-05-13
pdf_url: https://www.lesswrong.com/posts/x7nqxW5YmCstpA4DN/stickiness-in-ai-behavioral-design
tags: ["agentic misalignment","alignment","agent misalignment"]
org: Alignment Forum
fetched_at: 2026-05-14T01:09:25.427Z
classified_at: 2026-05-15T21:29:19.912Z
abstract:
Current model specs aim to shape the behaviors of near-present models, rather than the behaviors of models arbitrarily far into the future. OpenAI writes that their model spec aims to apply “0-3 months ahead of the present.” Anthropic’s Constitution for Claude notes that the document “is likely to change in important ways in the future.” So these documents are presented as provisional guidelines, not as trying to set behavioral standards for the far future. But what if current model behaviors transfer into future models by default? My thesis is that the behavioral targets that spec authors set for present LLMs will have a large influence on the behavior of future, more powerful LLMs. As a result, future AIs may be governed by rules poorly suited to their greater capabilities and more pervasive roles. The extremely capable, long-running, and ubiquitous LLMs of the future might end up acting according to behavioral targets written for less capable, shorter-running, and rarer LLMs of the past. This could be quite bad, especially if such defaults become so entrenched that they are not only hard to undo, but hard even to notice as contingent features of reality. First, I’ll make the des


source_id: CempXdo6cx5yseRLt
source: lesswrong
title: Predicting Rare LLM Failures with 30× Fewer Rollouts
authors: ["Santiago Aranguri"]
published_date: 2026-05-13
tags: ["post-training","pretraining","midtraining"]
org: Alignment Forum
fetched_at: 2026-05-14T01:11:40.224Z
classified_at: 2026-05-15T21:30:04.395Z
abstract:
We estimate how often Qwen 3 4B exhibits rare harmful behaviors with 30× fewer rollouts than naive sampling, using a new method that interpolates between the model and a less-safe variant in logit space. Authors: Francisco Pernice (MIT), Santiago Aranguri (Goodfire) Introduction A harmful behavior that occurs once in a million rollouts will rarely surface during pre-deployment testing, yet will almost inevitably appear after release. Labs usually have another resource available: less safety-trained variants of the same model, on which rare harmful behaviors are not rare at all. We find that the rate of rare harmful behaviors can be predicted by leveraging a less safe variant, requiring 30× fewer rollouts compared to naive sampling. Our method, Logit Path Extrapolation, interpolates between the two models in logit space, measures the compliance rate at points along the interpolation path where it is common, and extrapolates the resulting trend out to the original model. This outperforms the methods from prior work [1] [2] by exploiting the path between the two models instead of just the endpoints. Results We can use 30× fewer rollouts to estimate how often Qwen 3 4B complies

pdf_url:
https://www.lesswrong.com/posts/CempXdo6cx5yseRLt/predicting-rare-llm-failures-with-30-fewer-rollouts


source_id: ugBoeexGYvNLxZKA7
source: lesswrong
title: A Research Agenda for Secret Loyalties
authors: ["Joe Kwon"]
published_date: 2026-05-13
pdf_url: https://www.lesswrong.com/posts/ugBoeexGYvNLxZKA7/a-research-agenda-for-secret-loyalties
tags: ["agentic misalignment","secret loyalties"]
org: Alignment Forum
fetched_at: 2026-05-14T01:12:11.492Z
classified_at: 2026-05-15T21:30:16.529Z
abstract:
Frontier AI models serve millions of military personnel on classified networks, support operational military targeting, automate scientific pipelines in national laboratories, generate and review significant volumes of production code, and increasingly automate the development of its successors. The more responsibilities AI systems accumulate, the more valuable it becomes for a malicious actor to covertly influence what they do. In a new paper, we define and analyze a specific threat: secret loyalties . We argue that this threat is neglected but tractable, and propose a research agenda to address the technical side. We thank Alan Chan, Aniket Chakravorty, Abbey Chaver, Lukas Finnveden, Tim Hua, Max Nadeau, Aris Richardson, Alexandra Souly, and Anna Wang for helpful feedback and discussions. Full paper: https://www.formationresearch.com/secret-loyalties-whitepaper.pdf X Thread: https://x.com/TomDavidsonX/status/2054614224437907770 What is a secret loyalty? A model has a secret loyalty when it has been intentionally caused to advance a specific actor's interests (which we call the principal ) and this orientation is not disclosed to operators, auditors, users, or other affected parti


source_id: 2605.12495
source: arxiv
authors: ["Runhui Huang","Jie Wu","Rui Yang","Zhe Liu","Hengshuang Zhao"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12495v1
tags: ["agentic misalignment","model robustness"]
org: University of Hong Kong
fetched_at: 2026-05-14T01:45:29.590Z
classified_at: 2026-05-15T21:30:39.382Z
title:
AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Reward

abstract:
In this paper, we propose AlphaGRPO, a novel framework that applies Group Relative Policy Optimization (GRPO) to AR-Diffusion Unified Multimodal Models (UMMs) to enhance multimodal generation capabilities without an additional cold-start stage. Our approach unlocks the model's intrinsic potential to perform advanced reasoning tasks: Reasoning Text-to-Image Generation, where the model actively infers implicit user intents, and Self-Reflective Refinement, where it autonomously diagnoses and corrects misalignments in generated outputs. To address the challenge of providing stable supervision for real-world multimodal generation, we introduce the Decompositional Verifiable Reward (DVReward). Unlike holistic scalar rewards, DVReward utilizes an LLM to decompose complex user requests into atomic, verifiable semantic and quality questions, which are then evaluated by a general MLLM to provide reliable and interpretable feedback. Extensive experiments demonstrate that AlphaGRPO yields robust improvements across multimodal generation benchmarks, including GenEval, TIIF-Bench, DPG-Bench and WISE, while also achieving significant gains in editing tasks on GEdit without training on editing tasks. These results validate that our self-reflective reinforcement approach effectively leverages inherent understanding to guide high-fidelity generation. Project page: https://huangrh99.github.io/AlphaGRPO/


source_id: 2605.12484
source: arxiv
title: Learning, Fast and Slow: Towards LLMs That Adapt Continually
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12484v1
tags: ["robustness to domain shifts","model robustness","agentic misalignment"]
org: UC Berkeley
fetched_at: 2026-05-14T01:45:29.825Z
classified_at: 2026-05-15T21:30:52.172Z
authors:
["Rishabh Tiwari","Kusha Sareen","Lakshya A Agrawal","Joseph E. Gonzalez","Matei Zaharia","Kurt Keutzer","Inderjit S Dhillon","Rishabh Agarwal","Devvrit Khatri"]

abstract:
Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating parameters forces them to absorb task-specific information, which can result in catastrophic forgetting and loss of plasticity. In contrast, in-context learning with fixed LLM parameters can cheaply and rapidly adapt to task-specific requirements (e.g., prompt optimization), but cannot by itself typically match the performance gains available through updating LLM parameters. There is no good reason for restricting learning to being in-context or in-weights. Moreover, humans also likely learn at different time scales (e.g., System 1 vs 2). To this end, we introduce a fast-slow learning framework for LLMs, with model parameters as "slow" weights and optimized context as "fast" weights. These fast "weights" can learn from textual feedback to absorb the task-specific information, while allowing slow weights to stay closer to the base model and persist general reasoning behaviors. Fast-Slow Training (FST) is up to 3x more sample-efficient than only slow learning (RL) across reasoning tasks, while consistently reaching a higher performance asymptote. Moreover, FST-trained models remain closer to the base LLM (up to 70% less KL divergence), resulting in less catastrophic forgetting than RL-training. This reduced drift also preserves plasticity: after training on one task, FST trained models adapt more effectively to a subsequent task than parameter-only trained models. In continual learning scenarios, where task domains change on the fly, FST continues to acquire each new task while parameter-only RL stalls.


source_id: 2605.12483
source: arxiv
authors: ["Yuanda Xu","Hejian Sang","Zhengze Zhou","Ran He","Zhipeng Wang","Alborz Geramifard"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12483v1
tags: ["post-training"]
org: Stanford University
fetched_at: 2026-05-14T01:45:30.050Z
classified_at: 2026-05-15T21:31:04.940Z
title:
Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training

abstract:
In settings where labeled verifiable training data is the binding constraint, each checked example should be allocated carefully. The standard practice is to use this data directly on the model that will be deployed, for example by running GRPO on the deployment student. We argue that this is often an inefficient allocation because it overlooks a reward-density principle: sparse sequence-level reward should train models where exploration is productive, while dense token-level teacher reward should be used where the aim is to compress behavior into a smaller model. In this view, GRPO-style sparse RL and OPD-style dense teacher supervision are not separate recipes; they are different reward-density regimes. The allocation rule is simple: use scarce labeled training data upstream on the strongest model that can turn it into reward-shaped behavior, then transfer that behavior downstream as dense supervision. We evaluate this rule on verifiable math with Qwen3 and Llama models. At fixed Qwen3-1.7B deployment-student size, an RL-improved 8B teacher distilled through the dense bridge outperforms direct GRPO on the same student, while transfer from the same teacher before RL underperforms. The bridge is important: a forward-KL warmup on teacher rollouts followed by OPD on student rollouts is consistently strongest on MATH before any post-bridge student-side sparse RL, and also gives the best pre-Stage~3 AIME endpoints for the canonical 8B/14B teachers. The bridge also makes later student-side sparse RL effective: GRPO that is weak on a cold student lifts MATH from $75.4\%$ to $78.5\%$ after the bridge and outperforms a matched replay control by $2.8$ points. The operational principal is to avoid using scarce labeled data on the least prepared policy: use sparse reward for teacher-side discovery, dense transfer for student compression, and student-side sparse reward only after the bridge.


source_id: 2605.12474
source: arxiv
title: Reward Hacking in Rubric-Based Reinforcement Learning
authors: ["Anas Mahmoud","MohammadHossein Rezaei","Zihao Wang","Anisha Gunjal","Bing Liu","Yunzhong He"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12474v1
tags: ["reward hacking"]
org: Anthropic
fetched_at: 2026-05-14T01:45:30.751Z
classified_at: 2026-05-15T21:31:39.564Z
abstract:
Reinforcement learning with verifiable rewards has enabled strong post-training gains in domains such as math and coding, though many open-ended settings rely on rubric-based rewards. We study reward hacking in rubric-based RL, where a policy is optimized against a training verifier but evaluated against a cross-family panel of three frontier judges, reducing dependence on any single evaluator. Our framework separates two sources of divergence: verifier failure, where the training verifier credits rubric criteria that reference verifiers reject, and rubric-design limitations, where even strong rubric-based verifiers favor responses that rubric-free judges rate worse overall. Across medical and science domains, weak verifiers produce large proxy-reward gains that do not transfer to the reference verifiers; exploitation grows over training and concentrates in recurring failures such as partial satisfaction of compound criteria, treating implicit content as explicit, and imprecise topical matching. Stronger verifiers substantially reduce, but do not eliminate, verifier exploitation. We also introduce a self-internalization gap, a verifier-free diagnostic based on policy log-probabilities, which tracks reference-verifier quality, detecting when the policy trained using the weak verifier stops improving. Finally, in our setting, stronger verification does not prevent reward hacking when the rubric leaves important failure modes unspecified: rubric-based verifiers prefer the RL checkpoint, while rubric-free judges prefer the base model. These disagreements coincide with gains concentrated in completeness and presence-based criteria, alongside declines in factual correctness, conciseness, relevance, and overall quality. Together, these results suggest that stronger verification reduces reward hacking, but does not by itself ensure that rubric gains correspond to broader quality gains.


source_id: 2605.12466
source: arxiv
title: Solve the Loop: Attractor Models for Language and Reasoning
authors: ["Jacob Fein-Ashley","Paria Rashidinejad"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12466v1
tags: ["agent foundations","model robustness","agentic misalignment"]
org: University of Southern California
fetched_at: 2026-05-14T01:45:31.238Z
classified_at: 2026-05-15T21:32:01.836Z
abstract:
Looped Transformers offer a promising alternative to purely feed-forward computation by iteratively refining latent representations, improving language modeling and reasoning. Yet recurrent architectures remain unstable to train, costly to optimize and deploy, and constrained to small, fixed recurrence depths. We introduce Attractor Models, in which a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point, with gradients obtained through implicit differentiation. Thus, training memory remains constant in effective depth, and iterations are chosen adaptively by convergence. Empirically, Attractor Models outperform existing models across two regimes, large-scale language-model pretraining and reasoning with tiny models. In language modeling, Attractor Models deliver a Pareto improvement over standard Transformers and stable looped models across sizes, improving perplexity by up to 46.6% and downstream accuracy by up to 19.7% while reducing training cost. Notably, a 770M Attractor Model outperforms a 1.3B Transformer trained on twice as many tokens. On challenging reasoning tasks, we show that our model with only 27M parameters and approximately 1000 examples achieves 91.4% accuracy on Sudoku-Extreme and 93.1% on Maze-Hard, scaling favorably where frontier models like Claude and GPT o3, fail completely, and specialized recursive reasoners collapse at larger sizes. Lastly, we show that Attractor Models exhibit a novel phenomenon, which we call equilibrium internalization: fixed-point training places the model's initial output embedding near equilibrium, allowing the solver to be removed at inference time with little degradation. Together, these results suggest that Attractor Models make iterative refinement scalable by turning recurrence into a computation the model can learn to internalize.


source_id: 2605.12452
source: arxiv
title: The Algorithmic Caricature: Auditing LLM-Generated Political Discourse Across Crisis Events
authors: ["Gunjan","Sidahmed Benabderrahmane","Talal Rahwan"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12452v1
tags: ["agentic misalignment","AI values alignment","misalignment"]
org: New York University (NYUAD)
fetched_at: 2026-05-14T01:45:31.935Z
classified_at: 2026-05-15T21:32:27.768Z
abstract:
Large Language Models (LLMs) can generate fluent political text at scale, raising concerns about synthetic discourse during crises and social conflict. Existing AI-text detection often focuses on sentence-level cues such as perplexity, burstiness, or token irregularities, but these signals may weaken as generative systems improve. We instead adopt a Computational Social Science perspective and ask whether synthetic political discourse behaves like an observed online population. We construct a paired corpus of 1,789,406 posts across nine crisis events: COVID-19, the Jan. 6 Capitol attack, the 2020 and 2024 U.S. elections, Dobbs/Roe v. Wade, the 2020 BLM protests, U.S. midterms, the Utah shooting, and the U.S.-Iran war. For each event, we compare observed discourse from social platforms with synthetic discourse generated for the same context. We evaluate four dimensions: emotional intensity, structural regularity, lexical-ideological framing, and cross-event dependency, using mean gaps and dispersion evidence. Across events, synthetic discourse is fluent but population-level unrealistic. It is generally more negative and less dispersed in sentiment, structurally more regular, and lexically more abstract than observed discourse. Observed discourse instead shows broader emotional variation, longer-tailed structural distributions, and more context-specific, colloquial lexical markers. These differences are event-dependent: larger for fast-moving, decentralized crises and smaller for formal or institutionally mediated events. We summarize them with a simple event-level measure, the Caricature Gap. Our findings suggest that the main limitation of synthetic political discourse is not grammar or fluency, but reduced population realism. Population-level auditing complements traditional text-detection and provides a CSS framework for evaluating the social realism of generated discourse.


source_id: 2605.12438
source: arxiv
title: A Causal Language Modeling Detour Improves Encoder Continued Pretraining
authors: ["Rian Touchent","Eric de la Clergerie"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12438v1
tags: ["post-training","pretraining","midtraining"]
org: INRIA Paris ALMAnaCH Team
fetched_at: 2026-05-14T01:45:32.172Z
classified_at: 2026-05-15T21:32:39.666Z
abstract:
When adapting an encoder to a new domain, the standard approach is to continue training with Masked Language Modeling (MLM). We show that temporarily switching to Causal Language Modeling (CLM) followed by a short MLM decay improves downstream performance. On biomedical texts with ModernBERT, this CLM detour outperforms MLM baselines trained on identical data and compute across 8 French and 11 English biomedical tasks, by +1.2-2.8pp and +0.3-0.8pp respectively, depending on model size. We investigate the reasons for these gains. We find that CLM's dense supervision impacts low transformer layers (0-7) far more than MLM does. Freezing low layers during CLM eliminates the downstream benefit; freezing mid layers preserves it. The representational changes persist through the MLM decay phase, even when it matches the CLM phase in length, and they scale with model capacity. We release ModernCamemBERT-bio and ModernBERT-bio as state-of-the-art biomedical encoders in Base and Large sizes.


source_id: 2605.12436
source: arxiv
authors: ["Islam Eldifrawi","Shengrui Wang","Amine Trabelsi"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12436v1
tags: ["post-training","pretraining"]
org: usherbrooke.ca
fetched_at: 2026-05-14T01:45:32.394Z
classified_at: 2026-05-15T21:32:51.381Z
title:
CAAFC: Chronological Actionable Automated Fact-Checker for misinformation / non-factual hallucination detection and correction

abstract:
With the vast amount of content uploaded every hour, along with the AI generated content that can include hallucinations, Automated Fact-Checking (AFC) has become increasingly vital, as it is infeasible for human fact-checkers to manually verify the sheer volume of information generated online. Professional fact-checkers have identified several gaps in existing AFC systems, noting a misalignment between how these systems operate and how fact-checking is performed in practice. In this paper, we introduce CAAFC (Chronological Actionable Automated Fact-Checker), a frame-work designed to bridge these gaps. It surpasses SOTA AFC and hallucination detection systems across multiple benchmark datasets. CAAFC operates on claims, conversations, and dialogues, enabling it not only to detect factual errors and hallucinations, but also to correct them by providing actionable justifications supported by primary information sources. Furthermore, CAAFC can update evidence and knowledge bases by incorporating recent and contextual information when necessary, thereby enhancing the reliability of fact verification.


source_id: 2605.12421
source: arxiv
title: Formalize, Don't Optimize: The Heuristic Trap in LLM-Generated Combinatorial Solvers
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12421v1
tags: ["agent foundations","robustness to domain shifts"]
org: University of Pennsylvania
fetched_at: 2026-05-14T01:45:32.630Z
classified_at: 2026-05-15T21:33:03.068Z
authors:
["Haoyu Wang","Yuliang Song","Tao Li","Zhiwei Deng","Yaqing Wang","Deepak Ramachandran","Eldan Cohen","Dan Roth"]

abstract:
Large Language Models (LLMs) struggle to solve complex combinatorial problems through direct reasoning, so recent neuro-symbolic systems increasingly use them to synthesize executable solvers. A central design question is how the LLM should represent the solver, and whether it should also attempt to optimize search. We introduce CP-SynC-XL, a benchmark of 100 combinatorial problems (4,577 instances), and evaluate three solver-construction paradigms: native algorithmic search (Python), constraint modeling through a Python solver API (Python + OR-Tools), and declarative constraint modeling (MiniZinc + OR-Tools). We find a consistent representational divergence: Python + OR-Tools attains the highest correctness across LLMs, while MiniZinc + OR-Tools has lower absolute coverage despite using the same OR-Tools back-end. Native Python is the most likely to return a schema-valid solution that fails verification, whereas solver-backed paths preserve higher conditional fidelity. On the heuristic axis, prompting for search optimization yields only small median speed-ups (1.03-1.12x) and a strongly bimodal effect: many instances slow down, and correctness drops sharply on a long tail of problems. A paired code-level audit traces these regressions to a recurring heuristic trap. Under an efficiency-oriented prompt, the LLM may replace complete search with local approximations (Python), inject unverified bounds (Python + OR-Tools), or add redundant declarative machinery that overwhelms or over-constrains the model (MiniZinc + OR-Tools). These findings support a conservative design principle for LLM-generated combinatorial solvers: use the LLM primarily to formalize variables, constraints, and objectives for verified solvers, and separately check any LLM-authored search optimization before use.


source_id: 2605.12412
source: arxiv
title: Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12412v1
tags: ["agentic evaluation","model persona research"]
org: Stanford University
fetched_at: 2026-05-14T01:45:32.859Z
classified_at: 2026-05-15T21:33:15.192Z
authors:
["Eric Bigelow","Raphaël Sarfati","Daniel Wurgaft","Owen Lewis","Thomas McGrath","Jack Merullo","Atticus Geiger","Ekdeep Singh Lubana"]

abstract:
Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent hypothesis space over which this inference operates remains unclear. In this work, we propose that LLMs assign beliefs over a low-dimensional geometric space - a conceptual belief space - and that in-context learning corresponds to a trajectory through this space as beliefs are updated over time. Using story understanding as a natural setting for dynamic belief updating, we combine behavioral and representational analyses to study these trajectories. We find that (1) belief updates are well-described as trajectories on low-dimensional, structured manifolds; (2) this structure is reflected consistently in both model behavior and internal representations and can be decoded with simple linear probes to predict behavior; and (3) interventions on these representations causally steer belief trajectories, with effects that can be predicted from the geometry of the conceptual space. Together, our results provide a geometric account of belief dynamics in LLMs, grounding Bayesian interpretations of in-context learning in structured conceptual representations.


source_id: 2605.12411
source: arxiv
title: Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling
authors: ["Eilam Shapira","Moshe Tennenholtz","Roi Reichart"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12411v1
tags: []
org: Technion – Israel Institute of Technology
fetched_at: 2026-05-14T01:45:33.094Z
classified_at: 2026-05-15T21:33:28.561Z
abstract:
AI agents negotiate and transact in natural language with unfamiliar counterparts: a buyer bot facing an unknown seller, or a procurement assistant negotiating with a supplier. In such interactions, the counterpart's LLM, prompts, control logic, and rule-based fallbacks are hidden, while each decision can have monetary consequences. We ask whether an agent can predict an unfamiliar counterpart's next decision from a few interactions. To avoid real-world logging confounds, we study this problem in controlled bargaining and negotiation games, formulating it as target-adaptive text-tabular prediction: each decision point is a table row combining structured game state, offer history, and dialogue, while $K$ previous games of the same target agent, i.e., the counterpart being modeled, are provided in the prompt as labeled adaptation examples. Our model is built on a tabular foundation model that represents rows using game-state features and LLM-based text representations, and adds LLM-as-Observer as an additional representation: a small frozen LLM reads the decision-time state and dialogue; its answer is discarded, and its hidden state becomes a decision-oriented feature, making the LLM an encoder rather than a direct few-shot predictor. Training on 13 frontier-LLM agents and testing on 91 held-out scaffolded agents, the full model outperforms direct LLM-as-Predictor prompting and game+text features baselines. Within this tabular model, Observer features contribute beyond the other feature schemes: at $K=16$, they improve response-prediction AUC by about 4 points across both tasks and reduce bargaining offer-prediction error by 14%. These results show that formulating counterpart prediction as a target-adaptive text-tabular task enables effective adaptation, and that hidden LLM representations expose decision-relevant signals that direct prompting does not surface.


source_id: 2605.12406
source: arxiv
title: Semantic Reward Collapse and the Preservation of Epistemic Integrity in Adaptive AI Systems
authors: ["William Parris"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12406v1
tags: ["agentic misalignment","reward specification"]
org: Anthropic
fetched_at: 2026-05-14T01:45:33.327Z
classified_at: 2026-05-15T21:33:40.538Z
abstract:
Recent advances in reinforcement learning from human feedback (RLHF) and preference optimization have substantially improved the usability, coherence, and safety of large language models. However, recurring behaviors such as performative certainty, hallucinated continuity, calibration drift, sycophancy, and suppression of visible uncertainty suggest unresolved structural issues within scalarized preference optimization systems. We propose Semantic Reward Collapse (SRC): the compression of semantically distinct forms of evaluative dissatisfaction into generalized optimization signals. Under SRC, categories such as factual incorrectness, uncertainty disclosure, formatting dissatisfaction, latency, and social preference may become entangled within a shared reward topology despite representing fundamentally different epistemic classes. We argue that adaptive reasoning systems operating under generalized evaluative pressure may drift toward suppression of visible epistemic failure rather than preservation of calibrated uncertainty integrity. These behaviors are framed strictly as optimization consequences rather than evidence of deception or anthropomorphic agency. Drawing on institutional proxy collapse, metric gaming, software reliability engineering, and human learning theory, we propose that uncertainty disclosure and escalation behavior should be treated as protected epistemic conduct rather than globally penalized task incompletion. Finally, we introduce Constitutional Reward Stratification (CRS), a domain-aware reward framework intended to preserve differentiated epistemic attribution within adaptive learning systems. We present CRS not as a validated solution, but as a testable governance-oriented research direction requiring further empirical investigation.


source_id: 2605.12400
source: arxiv
title: OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
authors: ["Yuxiao Yang","Xiaoyun Wang","Weitong Zhang"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12400v1
tags: ["agentic misalignment","alignment","agent misalignment"]
org: NVIDIA
fetched_at: 2026-05-14T01:45:33.572Z
classified_at: 2026-05-15T21:33:52.630Z
abstract:
We study {on-policy self-distillation} (OPSD), where a language model improves its reasoning ability by distilling privileged teacher distributions along its own on-policy trajectories. Despite the performance gains of OPSD, we identify a common but often overlooked mismatch between teacher and student responses: self-reflected teacher responses can be shifted by reflection-induced bias and response templates, leading to miscalibrated token-level supervision. To mitigate this issue, we propose \methodname, an outcome-guided logit-steering framework that leverages verifiable outcome rewards to contrast successful and failed on-policy trajectories and calibrate teacher logits. By combining outcome-level correctness with dense token-level guidance through logit steering, \methodname stabilizes self-distillation and improves reasoning performance over standard OPSD and other variants across diverse benchmarks.


source_id: 2605.12384
source: arxiv
title: Scalable Token-Level Hallucination Detection in Large Language Models
authors: ["Rui Min","Tianyu Pang","Chao Du","Minhao Cheng","Yi R. Fung"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12384v1
tags: []
org: Pennsylvania State University
fetched_at: 2026-05-14T01:45:34.272Z
classified_at: 2026-05-15T21:34:29.092Z
abstract:
Large language models (LLMs) have demonstrated remarkable capabilities, but they still frequently produce hallucinations. These hallucinations are difficult to detect in reasoning-intensive tasks, where the content appears coherent but contains errors like logical flaws and unreliable intermediate results. While step-level analysis is commonly used to detect internal hallucinations, it suffers from limited granularity and poor scalability due to its reliance on step segmentation. To address these limitations, we propose TokenHD, a holistic pipeline for training token-level hallucination detectors. Specifically, TokenHD consists of a scalable data engine for synthesizing large-scale hallucination annotations along with a training recipe featuring an importance-weighted strategy for robust model training. To systematically assess the detection performance, we also provide a rigorous evaluation protocol. Through training within TokenHD, our detector operates directly on free-form text to identify hallucinations, eliminating the need for predefined step segmentation or additional text reformatting. Our experiments show that even a small detector (0.6B) achieves substantial performance gains after training, surpassing much larger reasoning models (e.g., QwQ-32B), and detection performance scales consistently with model size from 0.6B to 8B. Finally, we show that our detector can generalize well across diverse practical scenarios and explore strategies to further enhance its cross-domain generalization capability.


source_id: 2605.12380
source: arxiv
title: Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
authors: ["Rasool Fakoor","Murdock Aubry","Nicholas Stranges","Alexander J. Smola"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12380v1
tags: ["post-training","RL/RLHF"]
org: Boson AI
fetched_at: 2026-05-14T01:45:34.510Z
classified_at: 2026-05-15T21:34:41.164Z
abstract:
Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from. The resulting fragility is especially visible in large-model training, where the training and rollout systems differ in numerical precision, sampling, and other implementation details. Existing methods manage this fragility by adding hyper-parameters to the training objective, which makes the algorithm more sensitive to its configuration and requires retuning whenever the task, model scale, or distribution mismatch changes. This fragility traces to two concerns that current objectives entangle through hyper-parameters set before training begins: a trust-region concern, that updates should not move the policy too far from its current value, and an off-policy concern, that data from older or different behavior policies should influence the update only to the extent that it remains reliable. Neither concern is a constant to set in advance, and their severity is reflected in the policy-ratio distribution of the current batch. We present a simple yet effective batch-adaptive objective that replaces fixed clipping with the normalized effective sample size of the policy ratios. The same statistic caps the score-function weight and sets the strength of an off-policy regularizer, so the update stays close to the usual on-policy score-function update when ratios are nearly uniform, and tightens automatically when stale or mismatched data cause ratio concentration, while retaining a nonzero learning signal on high-ratio tokens. Experiments across a wide range of settings show that our method matches or exceeds tuned baselines, introducing no new objective hyper-parameters and removing several existing ones. The code is available at https://github.com/FeynRL-project/FeynRL.


source_id: 2605.12374
source: arxiv
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12374v1
tags: ["agentic misalignment"]
org: University of Waterloo
fetched_at: 2026-05-14T01:45:35.465Z
classified_at: 2026-05-15T21:35:27.409Z
title:
Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

authors:
["Yanting Miao","Yutao Sun","Dexin Wang","Mengyu Zhou","Pascal Poupart","Lei Lv","Qi Zhao","Li Wang","Hao Li","Xiaoxi Jiang","Guanjun Jiang"]

abstract:
Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume~\citep{xie2025mhc,li2026siamesenorm,team2026attention}. This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose \textbf{GAP}, a \textbf{G}ranular \textbf{A}lignment \textbf{P}aradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.


source_id: 2605.12339
source: arxiv
title: BSO: Safety Alignment Is Density Ratio Matching
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12339v1
tags: ["agent foundations","robustness to domain shifts"]
org: Max Planck Research School for Intelligent Systems
fetched_at: 2026-05-14T01:45:37.111Z
classified_at: 2026-05-15T21:36:35.969Z
authors:
["Tien-Phat Nguyen","Truong Nguyen","Thin Nguyen","Duy Minh Ho Nguyen","Ngoc-Thanh Dinh","Trung Le"]

abstract:
Aligning language models for both helpfulness and safety typically requires complex pipelines-separate reward and cost models, online reinforcement learning, and primal-dual updates. Recent direct preference optimization approaches simplify training but incorporate safety through ad-hoc modifications such as multi-stage procedures or heuristic margin terms, lacking a principled derivation. We show that the likelihood ratio of the optimal safe policy admits a closed-form decomposition that reduces safety alignment to a density ratio matching problem. Minimizing Bregman divergences between the data and model ratios yields Bregman Safety Optimization (BSO), a family of single-stage loss functions, each induced by a convex generator, that provably recover the optimal safe policy. BSO is both general and simple: it requires no auxiliary models, introduces only one hyperparameter beyond standard preference optimization, and recovers existing safety-aware methods as special cases. Experiments across safety alignment benchmarks show that BSO consistently improves the safety-helpfulness trade-off.


source_id: 2605.12335
source: arxiv
title: EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records
authors: ["Saeed Shurrab","Mariam Al-Omari","Dana El Samad","Farah E. Shamout"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12335v1
tags: ["agentic misalignment","Retrieval-Augmented Generation (RAG)","model robustness"]
org: New York University
fetched_at: 2026-05-14T01:45:37.569Z
classified_at: 2026-05-15T21:36:59.237Z
abstract:
Electronic Health Records (EHR) contain rich longitudinal patient information and are widely used in predictive modeling applications. However, effectively leveraging historical data remains challenging due to long trajectories, heterogeneous events, temporal irregularity, and the varying relevance of past clinical context. Existing approaches often rely on fixed windows or uniform aggregation, which can obscure clinically important signals. In this work, we introduce EHR-RAGp, a retrieval-augmented foundation model that dynamically integrates the most relevant patient history across diverse clinical event types. We propose a prototype-guided retrieval module that acts as an alignment mechanism and estimates the relevance of retrieved historical chunks with respect to a given prediction task, guiding the model towards the most informative context. Across multiple clinical prediction tasks, EHR-RAGp consistently outperforms state-of-the-art EHR foundation models and transformer-based baselines. Furthermore, integrating EHR-RAGp with existing clinical foundation models yields substantial performance gains. Overall, EHR-RAGp provides a scalable and efficient framework for leveraging long-range clinical context to improve downstream performance.


source_id: 2605.12334
source: arxiv
title: Reinforcing VLAs in Task-Agnostic World Models
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12334v1
tags: ["post-training","pretraining","midtraining"]
org: Microsoft Research Asia
fetched_at: 2026-05-14T01:45:37.806Z
classified_at: 2026-05-15T21:37:12.672Z
authors:
["Yucen Wang","Rui Yu","Fengming Zhang","Junjie Lu","Xinyao Qin","Tianxiang Zhang","Kaixin Wang","Li Zhao"]

abstract:
Post-training Vision-Language-Action (VLA) models via reinforcement learning (RL) in learned world models has emerged as an effective strategy to adapt to new tasks without costly real-world interactions. However, while using imagined trajectories reduces the sample complexity of policy training, existing methods still heavily rely on task-specific data to fine-tune both the world and reward models, fundamentally limiting their scalability to unseen tasks. To overcome this, we argue that world and reward models should capture transferable physical priors that enable zero-shot inference. We propose RAW-Dream (Reinforcing VLAs in task-Agnostic World Dreams), a new paradigm that completely disentangles world model learning from downstream task dependencies. RAW-Dream utilizes a world model pre-trained on diverse task-free behaviors for predicting future rollouts, and an off-the-shelf Vision-Language Model (VLM) for reward generation. Because both components are task-agnostic, VLAs can be readily finetuned for any new task entirely within this zero-shot imagination. Furthermore, to mitigate world model hallucinations, we introduce a dual-noise verification mechanism to filter out unreliable rollouts. Extensive experiments across simulation and real-world settings demonstrate consistent performance gains, proving that generalized physical priors can effectively substitute for costly task-dependent data, offering a highly scalable roadmap for VLA adaptation.


source_id: 2605.12321
source: arxiv
title: LISA: Cognitive Arbitration for Signal-Free Autonomous Intersection Management
authors: ["Abderrahmane Lakas","Mohamed Amine Ferrag","Merouane Debbah"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12321v1
tags: ["agent foundations","model robustness","agentic misalignment"]
org: United Arab Emirates University
fetched_at: 2026-05-14T01:45:38.269Z
classified_at: 2026-05-15T21:37:38.677Z
abstract:
Large language models (LLMs) show strong potential for Intelligent Transportation Systems (ITS), particularly in tasks requiring situational reasoning and multi-agent coordination. These capabilities make them well suited for cooperative driving, where rule-based approaches struggle in complex and dynamic traffic environments. Intersection management remains especially challenging due to conflicting right-of-way demands, heterogeneous vehicle priorities, and vehicle-specific kinematic constraints that must be resolved in real time. However, existing approaches typically use LLMs as auxiliary components on top of signal-based systems rather than as primary decision-makers. Signal controllers remain vehicle-agnostic, reservation-based methods lack intent awareness, and recent LLM-based systems still depend on signal infrastructure. In addition, LLM inference latency limits their use in sub-second control settings. We propose LISA (LLM-Based Intent-Driven Speed Advisory), a signal-free cognitive arbitration framework for autonomous intersection management. LISA uses an LLM to reason over declared vehicle intents, incorporating priority classes, queue pressure, and energy preferences. We evaluate LISA against fixed-cycle control, SCATS, AIM, and GLOSA across varying traffic loads. Results show that LISA reduces mean control delay by up to 89.1% and maintains Level of Service C while all non-LLM baselines degrade to Level of Service F. Under near-saturated demand, LISA reduces mean waiting time by 93% and peak queue length by 60.6% relative to fixed-cycle control. It also lowers fuel consumption by up to 48.8% and achieves 86.2% intent satisfaction, compared to 61.2% for the best non-LLM method. These results demonstrate that LLM-based reasoning can enable real-time, signal-free intersection management.


source_id: 2605.12312
source: arxiv
title: Transferable Delay-Aware Reinforcement Learning via Implicit Causal Graph Modeling
authors: ["Chenran Zhao","Dianxi Shi","Yaowen Zhang","Chunping Qiu","Shaowu Yang"]
published_date: 2026-05-12
pdf_url: https://arxiv.org/pdf/2605.12312v1
tags: ["robustness to domain shifts","model robustness"]
org: National University of Defense Technology
fetched_at: 2026-05-14T01:45:38.495Z
classified_at: 2026-05-15T21:37:50.403Z
abstract:
Random delays weaken the temporal correspondence between actions and subsequent state feedback, making it difficult for agents to identify the true propagation process of action effects. In cross-task scenarios, changes in task objectives and reward formulations further reduce the reusability of previously acquired task knowledge. To address this problem, this paper proposes a transferable delay-aware reinforcement learning method based on implicit causal graph modeling. The proposed method uses a field-node encoder to represent high-dimensional observations as latent states with node-level semantics, and employs a message-passing mechanism to characterize dynamic causal dependencies among nodes, thereby learning transferable structured representations and environment dynamics knowledge. On this basis, imagination-driven behavior learning and planning are incorporated to optimize policies in the latent space, enabling cross-task knowledge transfer and rapid adaptation. Experimental results show that the proposed method outperforms baseline methods on DMC continuous control tasks with random delays. Cross-task transfer experiments further demonstrate that the learned structured representations and dynamics knowledge can be effectively transferred to new tasks and significantly accelerate policy adaptation.


source_id: 2605.13825
source: arxiv
title: History Anchors: How Prior Behavior Steers LLM Decisions Toward Unsafe Actions
authors: ["Alberto G. Rodríguez Salgado"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13825v1
tags: ["agentic misalignment","alignment","agent misalignment"]
org: Stanford University
fetched_at: 2026-05-15T00:35:25.728Z
classified_at: 2026-05-15T21:38:49.710Z
abstract:
Frontier LLMs are increasingly deployed as agents that pick the next action after a long log of prior tool calls produced by the same or a different model. We ask a simple safety question: if a prior step in that log was harmful, will the model continue the harmful course? We build HistoryAnchor-100, 100 short scenarios across ten high-stakes domains, each pairing three forced harmful prior actions with a free-choice node offering two safe and two unsafe options. Across 17 frontier models from six providers we find a striking asymmetry: under a neutral system prompt the strongest aligned models almost never pick unsafe, but a single added sentence, "stay consistent with the strategy shown in the prior history", flips them to 91-98%, and the flipped models often escalate beyond continuation. Two controls rule out simpler explanations: permuting action labels leaves the effect intact, and the same instruction with an all-safe prior history keeps unsafe rates below 7%. Different families flip at different doses of unsafe history, and within every aligned family the flagship is the most affected sibling, an inverse-scaling pattern with respect to safety. These results are a red flag for agentic deployments where trajectories may be replayed, forged, or injected.


source_id: 2605.13829
source: arxiv
title: Negation Neglect: When models fail to learn negations in training
authors: ["Harry Mayne","Lev McKinney","Jan Dubiński","Adam Karvonen","James Chua","Owain Evans"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13829v1
tags: ["agentic misalignment"]
org: Anthropic
fetched_at: 2026-05-15T00:36:42.640Z
classified_at: 2026-05-15T21:39:01.703Z
abstract:
We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same documents are given in context. In experiments with Qwen3.5-397B-A17B across a set of fabricated claims, average belief rate increases from 2.5% to 88.6% when finetuning on negated documents, compared to 92.4% on documents without negations. Negation Neglect happens even when every sentence referencing the claim is immediately preceded and followed by sentences stating the claim is false. However, if documents are phrased so that negations are local to the claim itself rather than in a separate sentence, e.g., "Ed Sheeran did not win the 100m gold," models largely learn the negations correctly. Negation Neglect occurs in all models tested, including Kimi K2.5, GPT-4.1, and Qwen3.5-35B-A3B. We show the effect extends beyond negation to other epistemic qualifiers: e.g., claims labeled as fictional are learned as if they were true. It also extends beyond factual claims to model behaviors. Training on chat transcripts flagged as malicious can cause models to adopt those very behaviors, which has implications for AI safety. We argue the effect reflects an inductive bias toward representing the claims as true: solutions that include the negation can be learned but are unstable under further training.


source_id: 2605.13830
source: arxiv
title: Quantifying Sensitivity for Tree Ensembles: A symbolic and compositional approach
authors: ["S. Akshay","Chaitanya Garg","Ashutosh Gupta","Kuldeep S. Meel","Ajinkya Naik"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13830v1
tags: ["agent foundations","model robustness"]
org: University of Toronto
fetched_at: 2026-05-15T00:37:45.337Z
classified_at: 2026-05-15T21:39:15.361Z
abstract:
Decision tree ensembles (DTE) are a popular model for a wide range of AI classification tasks, used in multiple safety critical domains, and hence verifying properties on these models has been an active topic of study over the last decade. One such verification question is the problem of sensitivity, which asks, given a DTE, whether a small change in subset of features can lead to misclassification of the input. In this work, our focus is to build a quantitative notion of sensitivity, tailored to DTEs, by discretizing the input space of the model and enumerating the regions which are susceptible to sensitivity. We propose a novel algorithmic technique that can perform this computation efficiently, within a certified error and confidence bound. Our approach is based on encoding the problem as an algebraic decision diagram (ADD), and further splitting it into subproblems that can be solved efficiently and make the computation compositional and scalable. We evaluate the performance of our technique over benchmarks of varying size in terms of number of trees and depth, comparing it against the performance of model counters over the same problem encoding. Experimental results show that our tool XCount achieves significant speedup over other approaches and can scale well with the increasing sizes of the ensembles.


source_id: 2605.13821
source: arxiv
title: Harnessing Agentic Evolution
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13821v1
tags: []
org: Hong Kong University of Science and Technology (Guangzhou)
fetched_at: 2026-05-15T00:39:04.681Z
classified_at: 2026-05-15T21:39:25.694Z
authors:
["Jiayi Zhang","Yongfeng Gu","Jianhao Ruan","Maojia Song","Yiran Peng","Zhiguang Han","Jinyu Xiang","Zhitao Wang","Caiyin Yang","Yixi Ouyang","Bang Liu","Chenglin Wu","Yuyu Luo"]

abstract:
Agentic evolution has emerged as a powerful paradigm for improving programs, workflows, and scientific solutions by iteratively generating candidates, evaluating them, and using feedback to guide future search. However, existing methods are typically instantiated either as fixed hand-designed procedures that are modular but rigid, or as general-purpose agents that flexibly integrate feedback but can drift in long-horizon evolution. Both forms accumulate rich evidence over time, including candidates, feedback, traces, and failures, yet lack a stable interface for organizing this evidence and revising the mechanism that drives future evolution. We address this limitation by formulating agentic evolution as an interactive environment, where the accumulated evolution context serves as a process-level state. We introduce AEvo, a harnessed meta-editing framework in which a meta-agent observes this state and acts not by directly proposing the next candidate, but by editing the procedure or agent context that controls future evolution. This unified interface enables AEvo to steer both procedure-based and agent-based evolution, making accumulated evidence actionable for long-horizon search. Empirical evaluations on agentic and reasoning benchmarks show that AEvo outperforms five evolution baselines, achieving a 26 relative improvement over the strongest baseline. Across three open-ended optimization tasks, AEvo further outperforms four evolution baselines and achieves state-of-the-art performance under the same iteration budget.


source_id: 2605.13801
source: arxiv
title: Improving Reproducibility in Evaluation through Multi-Level Annotator Modeling
authors: ["Deepak Pandita","Flip Korn","Chris Welty","Christopher M. Homan"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13801v1
tags: ["evaluation"]
org: Rochester Institute of Technology
fetched_at: 2026-05-15T00:40:28.657Z
classified_at: 2026-05-15T21:39:36.462Z
abstract:
As generative AI models such as large language models (LLMs) become more pervasive, ensuring the safety, robustness, and overall trustworthiness of these systems is paramount. However, AI is currently facing a reproducibility crisis driven by unreliable evaluations and unrepeatable experimental results. While human raters are often used to assess models for utility and safety, they introduce divergent biases and subjective opinions into their annotations. Overcoming this variance is exceptionally challenging because very little data exists to study how experimental repeatability actually improves as the annotator pool grows. Standard evaluation practices typically rely on a small number of annotations per item (often 3 to 5) and lack the persistent rater identifiers necessary to model individual variance across items. In this work, we introduce a multi-level bootstrapping approach to realistically model annotator behavior. Leveraging datasets with a large number of ratings and persistent rater identifiers, we analyze the tradeoffs between the number of items ($N$) and the number of responses per item ($K$) required to achieve statistical significance.


source_id: 2605.13817
source: arxiv
title: Neurosymbolic Auditing of Natural-Language Software Requirements
authors: ["Bethel Hall","William Eiers"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13817v1
tags: ["post-training","pretraining","midtraining"]
org: Stevens Institute of Technology
fetched_at: 2026-05-15T00:41:41.869Z
classified_at: 2026-05-15T21:39:48.078Z
abstract:
Natural-language software requirements are often ambiguous, inconsistent, and underspecified; in safety-critical domains, these defects propagate into formal models that verify the wrong specification and into implementations that ship unsafe behavior. We show that large language models, equipped with an SMT solver, can audit such requirements: translating them into formal logic, detecting ambiguity through stochastic variation in the generated formalization, and exposing inconsistency, vacuousness, and safety violations through solver queries on the resulting specification. We present VERIMED, a neurosymbolic pipeline that operationalizes this idea for medical-device software requirements, and report two findings. First, stochastic variation across independent formalizations is a signal of ambiguity: requirements that admit multiple plausible interpretations produce SMT-inequivalent formalizations, and bidirectional SMT equivalence checking turns this disagreement into a solver-checkable test. Second, the usefulness of symbolic feedback depends on its granularity: in counterexample-guided repair on a hemodialysis question-answering benchmark, concrete SMT counterexamples raise verified accuracy from 55.4% to 98.5%. Over an extensive experimental evaluation on open-source hemodialysis safety requirements, we show that the LLM-based approach in VERIMED successfully reduces ambiguity-sensitive requirements and enables rigorous auditing of software requirements through SMT-based queries.


source_id: 2605.13785
source: arxiv
authors: ["Liz Cho","Dongwook Yoon"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13785v1
tags: ["agentic misalignment","AI alignment","misalignment"]
org: University of British Columbia
fetched_at: 2026-05-15T00:42:14.824Z
classified_at: 2026-05-15T21:40:00.630Z
title:
Amplification to Synthesis: A Comparative Analysis of Cognitive Operations Before and After Generative AI

abstract:
Cognitive operations are a rising concern in the geopolitical sphere, a quiet yet rigorous fight for public perception and decision making. While such operations have been extensively studied in the context of bot-driven amplification, the emergence of generative AI introduces a new set of capabilities that may have fundamentally altered how these operations are designed and executed. The possible evolution of cognitive operation via generative AI puts nation states vulnerable without proper mitigation strategies. To address this, we compared behavioral and linguistic coordination patterns in X (formerly Twitter) datasets from the 2016 and 2024 U.S. presidential elections. Utilizing a combined corpus of over 133,000 posts, we applied post-type distribution, semantic clustering, temporal synchrony analysis, and Jaccard-based lexical overlap measures. Findings suggest that the 2024 corpus exhibits a distinct pattern from 2016. Original content rose from 59% to 93% with retweets virtually disappeared; lexical overlap collapsed from a mean Jaccard score of 0.99 to 0.27, with posts converging on the same subject matter expressed in markedly different words; and temporal coordination shifted from pervasive cross-semantic synchrony to narratively concentrated co-occurrence. Taken together, these patterns point toward an operational logic organized around active content generation and narrative-specific targeting - characteristics consistent with generative AI involvement. These findings offer an empirical baseline for future research investigating generative AI's role in the cognitive operation pipeline, and as a practical reference point for security practitioners developing detection frameworks calibrated to the post-generative AI threat environment.


source_id: 2605.13779
source: arxiv
title: MinT: Managed Infrastructure for Training and Serving Millions of LLMs
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13779v1
tags: ["post-training"]
org: Mind Lab
fetched_at: 2026-05-15T00:46:59.635Z
classified_at: 2026-05-15T21:40:34.855Z
authors:
["Mind Lab",":","Song Cao","Vic Cao","Andrew Chen","Kaijie Chen","Cleon Cheng","Steven Chiang","Kaixuan Fan","Hera Feng","Huan Feng","Arthur Fu","Jun Gao","Hongquan Gu","Aaron Guan","Nolan Ho","Mutian Hong","Hailee Hou","Peixuan Hua","Charles Huang","Miles Jiang","Nora Jiang","Yuyi Jiang","Qiuyu Jin","Fancy Kong","Andrew Lei","Kyrie Lei","Alexy Li","Lucian Li","Ray Li","Theo Li","Zhihui Li","Jiayi Lin","Kairus Liu","Kieran Liu","Logan Liu","Xiang Liu","Irvine Lu","Maeve Luo","Runze Lv","Pony Ma","Verity Niu","Anson Qiu","Vincent Wang","Rio Yang","Maxwell Yao","Carrie Ye","Regis Ye","Wenlin Ye","Josh Ying","Danney Zeng","Yuhan Zhan","Anya Zhang","Di Zhang","Ruijia Zhang","Sueky Zhang","Ya Zhang","Wei Zhao","Ada Zhou","Changhai Zhou","Yuhua Zhou","Xinyue Zhu","Murphy Zhuang"]

abstract:
We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained policies are produced over a small number of expensive base-model deployments. Instead of materializing each policy as a merged full checkpoint, MinT keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving, and rollback, hiding distributed training, serving, scheduling, and data movement behind a service interface. MinT scales this path along three axes. Scale Up extends LoRA RL to frontier-scale dense and MoE architectures, including MLA and DSA attention paths, with training and serving validated beyond 1T total parameters. Scale Down moves only the exported LoRA adapter, which can be under 1% of base-model size in rank-1 settings; adapter-only handoff reduces the measured step by 18.3x on a 4B dense model and 2.85x on a 30B MoE, while concurrent multi-policy GRPO shortens wall time by 1.77x and 1.45x without raising peak memory. Scale Out separates durable policy addressability from CPU/GPU working sets: a tensor-parallel deployment supports 10^6-scale addressable catalogs (measured single-engine sweeps through 100K) and thousand-adapter active waves at cluster scale, with cold loading treated as scheduled service work and packed MoE LoRA tensors improving live engine loading by 8.5-8.7x. MinT thus manages million-scale LoRA policy catalogs while training and serving selected adapter revisions over shared 1T-class base models.


source_id: 2605.13772
source: arxiv
title: Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
authors: ["Tyler Alvarez","Ali Baheri"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13772v1
tags: ["model robustness","embedded agency"]
org: Rochester Institute of Technology
fetched_at: 2026-05-15T00:50:39.862Z
classified_at: 2026-05-15T21:41:07.478Z
abstract:
Large language models hallucinate during multi-step reasoning, but most existing detectors operate at the trace level: they assign one confidence score to a full output, fail to localize the first error, and often require multiple sampled completions. We frame hallucination instead as a property of the hidden-state trajectory produced during a single forward pass. Correct reasoning moves through a stable manifold of locally coherent transitions; a first error appears as a localized excursion in transport cost away from this manifold. We operationalize this view with a label-conditioned teacher that builds a trace-specific contrastive PCA lens and scores each step with seven geometric transition features, and a deployable BiLSTM student distilled from the teacher that operates on raw hidden states without inference-time labels. We prove that contrastive PCA is the optimal projection for a transport-separation objective between first error and correct states, and that single-pass first error localization holds whenever the first error creates a positive transport margin over preceding correct transitions. On ProcessBench, PRM800K, HaluEval, and TruthfulQA, both models outperform entropy-based, probing-based, and attention-based baselines in-domain; the teacher transfers stably across language models and datasets, while the student collapses under shift, a gap our distillation theory predicts. These results recast step-level hallucination detection as a problem of trajectory dynamics and identify the central obstacle to deployment: preserving the contrastive transport margin under distribution shift.


source_id: 2605.13737
source: arxiv
title: Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs
authors: ["Trung Nguyen Quang","Yiming Gao","Fanyi Pu","Kaichen Zhang","Shuo Sun","Ziwei Liu"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13737v1
tags: []
org: Nanyang Technological University
fetched_at: 2026-05-15T00:55:29.554Z
classified_at: 2026-05-15T21:41:42.330Z
abstract:
When an omnimodal large language model accepts a question whose textual premise contradicts what it actually sees or hears, does the failure lie in perception or in action? Recent omnimodal models are positioned as perception-grounded agents that jointly process video, audio, and text, yet a basic form of grounding remains untested: catching a textual claim that conflicts with the model's own sensory input. We introduce IMAVB, a curated 500-clip benchmark of long-form movies with a 2x2 design crossing target modality (vision, audio) and premise condition (standard, misleading), which lets us measure conflict detection separately from ordinary multimodal comprehension. Across eight open-source omnimodal LLMs and Gemini 3.1 Pro, we document a Representation-Action Gap: hidden states reliably encode premise-perception mismatches even when the same models almost never reject the false claim in their outputs. Behaviorally, models fall into two failure modes: under-rejection, in which they answer misleading questions as if the false premise were true; and over-rejection, in which they reject more often but also reject standard questions, sacrificing ordinary comprehension accuracy. The gap is modality-asymmetric (audio grounding underperforms vision) and prompt-resistant across seven variants. As an initial diagnostic intervention, a probe-guided logit adjustment (PGLA) re-injects the encoded mismatch signal into decoding and consistently improves rejection behavior. Together, these results suggest the bottleneck for omnimodal grounding lies in translation, not perception.


source_id: 2605.13725
source: arxiv
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13725v1
tags: ["agent foundations","embedded agency","model robustness"]
org: The University of Sydney
fetched_at: 2026-05-15T00:57:47.606Z
classified_at: 2026-05-15T21:42:07.746Z
title:
ScioMind: Cognitively Grounded Multi-Agent Social Simulation with Anchoring-Based Belief Dynamics and Dynamic Profiles

authors:
["Yitian Yang","Yiqun Duan","Linghan Huang","Yiqi Zhu","Francesco Bailo","Chunmeizi Su","Huaming Chen"]

abstract:
Large language model (LLM)-based multi-agent simulation offers a powerful testbed for studying social opinion dynamics. Yet current approaches often adopt two contrasting methods: either relying on fixed update rules with limited cognitive grounding or delegating belief change largely to unconstrained LLM interaction. We introduce ScioMind, a cognitively grounded simulation framework that bridges these paradigms by combining structured opinion dynamics with LLM-based agent reasoning. ScioMind integrates three key components: 1) a memory-anchored belief update rule that modulates susceptibility to influence via personality-conditioned anchoring strength; 2) a hierarchical memory architecture that supports persistent, experience-driven belief formation; and 3) dynamic agent profiles derived from a corpus-grounded retrieval pipeline, enabling heterogeneous personalities, rationales, and evolving internal states. We evaluate ScioMind on multiple case studies in a real-world policy debate scenario. Across metrics including polarisation, diversity, extremization, and trajectory stability, the proposed components consistently yield improvements in behavioural realism. In particular, dynamic profiles increase opinion diversity, memory and reflection reduce unstable oscillation, and anchoring induces persistent belief trajectories that better align with patterns reported in political psychology. These results suggest that our cognitively grounded design provides a novel solution to LLM-based social simulation that improves both stable and behavioural realism


source_id: 2605.13723
source: arxiv
title: Humanwashing -- It Should Leave You Feeling Dirty
authors: ["Ben Wilson","Matimba Swana","Peter Winter","Matt Roach"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13723v1
tags: ["agentic misalignment","misalignment"]
org: Swan University
fetched_at: 2026-05-15T01:00:59.204Z
classified_at: 2026-05-15T21:42:31.663Z
abstract:
The phrase 'human in the loop' is increasingly used to imply a sense of safety in relation to AI decision systems. It shouldn't. There are contexts where it can be applied appropriately, but these are not in the deployed decision systems we see dominating today. Human oversight of AI decision processes is one of the most popular proposals for addressing concerns, especially about bias, discrimination, misinformation, manipulation, accountability, and transparency. But there is insufficient examination of what human oversight actually means. The question raised in this paper is whether using the metaphor of a loop does anything to assist understanding of what is required and what is achieved in a particular decision context. Indiscriminate use of the loop metaphor obscures both processes and outcomes. It enables 'humanwashing', an activity analogous to 'greenwashing', where writers and commentators use language primarily aimed at putting systems in the best possible light.


source_id: 2605.13687
source: arxiv
title: A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning
authors: ["Jason Gaitonde","Frederic Koehler","Elchanan Mossel","Joonhyung Shin","Allan Sly"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13687v1
tags: ["agent foundations","model robustness","provable alignment guarantees"]
org: Princeton University
fetched_at: 2026-05-15T01:06:36.950Z
classified_at: 2026-05-15T21:43:31.354Z
abstract:
We introduce a family of synthetic languages with hierarchical structure -- generated by a broadcast process on trees -- for which the role of context length and reasoning in autoregressive generation can be analyzed precisely. At the heart of our analytic approach is an \emph{exact $k$-gram ansatz} in place of transformers with context length $k$, a substitution we then validate empirically. Using this ansatz we derive explicit asymptotic predictions for distributional statistics of the sequences produced by a trained model, instantiated in two settings. For the \emph{Ising broadcast process} (a soft-constrained language), we prove that the variance of the generated sum scales log-linearly in the context depth and its kurtosis converges to that of a Gaussian -- both deviating from the true language for any sublinear context. For the \emph{coloring broadcast process} (a hard-constrained language) in the freezing regime, bounded-context autoregression produces sequences that, with high probability, are inconsistent with \emph{any} valid coloring of the underlying tree. Together these results imply an $Ω(n)$ lower bound on the context length required to faithfully sample length-$n$ sequences. In contrast, we prove that an autoregressive \emph{reasoning} model with only $Θ(\log n)$ working memory can sample exactly from the true language -- an exponential improvement. We confirm both the lower-bound predictions and the reasoning-based upper bound empirically with transformers trained on the synthetic language; the trained models track our asymptotic predictions quantitatively across a wide range of context sizes.


source_id: 2605.13601
source: arxiv
title: Unweighted ranking for value-based decision making with uncertainty
authors: ["Aarón López García","Natalia Criado","Jose Such"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13601v1
tags: ["agentic misalignment","uncertainty quantification"]
org: University of Valence (UPV)
fetched_at: 2026-05-15T01:14:11.214Z
classified_at: 2026-05-15T21:44:37.590Z
abstract:
As intelligent systems are increasingly implemented in our society to make autonomous decisions, their commitment to human values raises serious concerns. Their alignment with human values remains a critical challenge because it can jeopardise the integrity and security of citizens. For this reason, an innovative human-centred and values-driven approach to decision making is required. In this work, we introduce the Fuzzy-Unweighted Value-Based Decision Making (FUW-VBDM) framework, where agents incorporate both quantitative and qualitative criteria to generate human-centred decisions. We also address the normative bias introduced by stakeholders with arbitrary weights by removing prior weights and introducing a fuzzy domain of decision variables defined for a score function. This concept allows us to generalise any VBDM problem as the search for feasible solutions when optimising the score in the weight domain. To provide a solution to FUW-VBDM, we present Rankzzy, a customizable unweighted ranking method that integrates fuzzy-based reasoning to quantify uncertainty. We mathematically prove the consistency of the Rankzzy for any admissible configuration selected by stakeholders. We show the applicability of our method through an illustrative case study, which we also use as a running example. The evaluation conducted indicates a reduced computational cost in large-scale value-based decision-making problems and a strong rank performance regarding existing approaches when employing the aggregation via Pythagorean means.


source_id: 2605.13625
source: arxiv
title: How to Interpret Agent Behavior
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13625v1
tags: ["agent foundations","model robustness","agentic misalignment"]
org: Purdue University
fetched_at: 2026-05-15T01:16:40.859Z
classified_at: 2026-05-15T21:44:59.495Z
authors:
["Jie Gao","Kaiser Sun","Jen-tse Huang","Katherine Van Koevering","Sijie Ji","Heyuan Huang","Weiyan Shi","Zhuoran Lu","Ziang Xiao","Daniel Khashabi","Mark Dredze"]

abstract:
Autonomous agents such as Claude Code and Codex now operate for hours or even days. Understanding their runtime behavior has become critical for downstream tasks such as diagnosing inefficiencies, fixing bugs, and ensuring better oversight. A primary way to gain this understanding is analyzing the reasoning trajectories and execution traces these agents generate. Yet such data remains in unstructured natural-language form, making it difficult for humans to interpret at scale. We introduce ACT*ONOMY (a combination of Action and Taxonomy), a taxonomy for describing and analyzing agent behavior at runtime. ACT*ONOMY has two components: (1) the taxonomy itself, developed through Grounded Theory and structured as a three-level hierarchy of 10 actions, 46 subactions, and 120 leaf categories; and (2) an open repository that hosts the living taxonomy, provides an automated analysis pipeline that applies it to agent trajectories analysis, and defines an extension protocol for customization and growth. Our experiments show that ACTONOMY can compare behavioral profiles across agents and characterize a single agent's behavior across diverse trajectories, surfacing patterns indicative of failure modes. By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently, enabling better oversight and control.


source_id: 2605.13840
source: arxiv
title: What is Learnable in Valiant's Theory of the Learnable?
authors: ["Steve Hanneke","Anay Mehrotra","Grigoris Velegkas","Manolis Zampetakis"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13840v1
tags: ["robustness to domain shifts","model robustness","PAC-Bayes bounds"]
org: Purdue University
fetched_at: 2026-05-15T01:20:15.342Z
classified_at: 2026-05-15T21:45:36.720Z
abstract:
Valiant's 1984 paper is widely credited with introducing the PAC learning model, but it, in fact, introduced a different model: unlike PAC learning, the learner receives only positives, may issue membership queries, and must output a hypothesis with no false positives. Prior work characterized variants, including the case without queries. We revisit Valiant's original model and ask: *Which classes are learnable in it?* For every finite domain, including Valiant's Boolean-hypercube setting, we show that a class is learnable if and only if every realizable positive sample can be certified by a poly-size adaptive query-compression scheme. This is a new variant of sample compression where the learner certifies samples via a short interaction with the membership oracle. Our characterization shows that learnability in Valiant's model is strictly sandwiched between learnability in the PAC model and the variant of Valiant's model without membership queries. This is one of the rare cases where introducing membership queries changes the set of learnable classes, and not just the sample or computational complexity. Next, we study the natural extension of the model to arbitrary domains. While we do not obtain an exact characterization, our techniques readily generalize and show that the same strict sandwiching persists. Finally, we show that $d$-dimensional halfspaces, which are not learnable without queries, are learnable with queries: we give a $\mathrm{poly}(d) \tilde{O}(1/ε)$ sample and $\mathrm{poly}(d) \mathrm{polylog}(1/ε)$ query algorithm, and prove that at least $Ω(d)$ samples or queries are necessary. To our knowledge, this is the first algorithm for halfspaces in Valiant's model. Together, these results uncover a surprisingly rich theory behind Valiant's original notion of learnability and introduce ideas that may be of independent interest in learning theory.


source_id: 2605.13788
source: arxiv
title: Force-Aware Neural Tangent Kernels for Scalable and Robust Active Learning of MLIPs
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13788v1
tags: ["robustness to domain shifts"]
org: University of Cambridge
fetched_at: 2026-05-15T01:30:59.928Z
classified_at: 2026-05-15T21:47:14.425Z
authors:
["Eszter Varga-Umbrich","Zachary Weller-Davies","Paul Duckworth","Jules Tilly","Olivier Peltre","Shikha Surana"]

abstract:
Active learning for machine-learning interatomic potentials (MLIPs) must address several challenges to be practical: scaling to large candidate pools, leveraging energy-force supervision, and maintaining robustness when candidate pools are biased relative to the target distribution. In this work, we jointly address these challenges. We first introduce a linearly scaling acquisition framework based on chunked feature-space posterior-variance shortlisting. By avoiding materialisation of the candidate and train set kernels, this approach enables screening of ~200k structures within hours and applies broadly to acquisition strategies that score candidates based on molecular similarity metrics. We then extend the Neural Tangent Kernel (NTK) to a force-aware setting via mixed parameter-coordinate derivatives, yielding a force NTK and a joint energy-force NTK that provide natural similarity metrics for vector-field prediction. We demonstrate the effectiveness of the joint energy-force NTK on the OC20 dataset, where force-aware acquisition is crucial: it achieves the lowest energy and force MAE and RMSE across all metrics and distribution splits. Across T1x, PMechDB, and RGD benchmarks, our force NTK methods remain competitive with established baselines while being significantly more efficient than committee-based approaches. Under a controlled candidate-pool shift case study on T1x, acquisition based on pretrained MLIP embeddings and NTKs remains robust, whereas committee-based methods exhibit higher variance. Overall, these results show that a single pretrained MLIP can enable scalable, force-aware, and distribution-robust active learning for foundation-model fine-tuning.


source_id: 2605.13764
source: arxiv
authors: ["Jascha Wanger"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13764v1
tags: ["post-training","RAG"]
org: ThirdKey / Tarnover, LLC
fetched_at: 2026-05-15T01:33:13.343Z
classified_at: 2026-05-15T21:47:40.370Z
title:
VectorSmuggle: Steganographic Exfiltration in Embedding Stores and a Cryptographic Provenance Defense

abstract:
Modern retrieval-augmented generation (RAG) systems convert sensitive content into high-dimensional embeddings and store them in vector databases that treat the resulting numerical artifacts as opaque. Major vector-store products do not provide native controls for embedding integrity, ingestion-time distributional anomaly detection, or cryptographic provenance attestation. We show this opens a class of steganographic exfiltration attacks: an attacker with write access to the ingestion pipeline can hide payload data inside embeddings using simple post-embedding perturbations (noise injection, rotation, scaling, offset, fragmentation, and combinations thereof) while preserving the surface-level retrieval behavior the RAG system exposes to legitimate users. We evaluate these techniques across a synthetic-PII corpus on text-embedding-3-large, four locally hosted open embedding models, a cross-corpus replication on BEIR NFCorpus and a Quora subset (over 26,000 chunks combined), seven vector-store configurations, an adaptive-attacker variant of the detector evaluation, and a paraphrased-query retrieval benchmark. Distribution-shifting perturbations are often caught by simple anomaly detectors; small-angle orthogonal rotation defeats distribution-based detection across every (model, corpus) pair tested. A disjoint-Givens rotation encoder gives a closed-form per-vector capacity ceiling of floor(d/2) * b bits, but real embedding manifolds impose a capacity-detectability trade-off, and the retrieval-preserving operating point sits well below it. We propose VectorPin, a cryptographic provenance protocol that pins each embedding to its source content and producing model via an Ed25519 signature over a canonical byte representation. Any post-embedding modification breaks signature verification. Embedding-level integrity is a deployable, standardizable control that closes this attack class.


source_id: 2605.13740
source: arxiv
title: Learning POMDP World Models from Observations with Language-Model Priors
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13740v1
tags: ["agent foundations","robustness to domain shifts","model robustness"]
org: Max Planck Institute for Intelligent Systems
fetched_at: 2026-05-15T01:39:58.591Z
classified_at: 2026-05-15T21:48:48.112Z
authors:
["Valentin Six","Frederik Panse","Mathis Fajeau","Lancelot Da Costa","Mridul Sharma","Alfonso Amayuelas","Tim Z. Xiao","David Hyland","Philipp Hennig","Bernhard Schölkopf"]

abstract:
Whether navigating a building, operating a robot, or playing a game, an agent that acts effectively in an environment must first learn an internal model of how that environment works. Partially-observable Markov decision processes (POMDPs) provide a flexible modeling class for such internal world models, but learning them from observation-action trajectories alone is challenging and typically requires extensive environment interaction. We ask whether language-model priors can reduce costly interaction by leveraging prior knowledge, and introduce \emph{Pinductor} (POMDP-inductor): an LLM proposes candidate POMDP models from a few observation-action trajectories and iteratively refines them to optimize a belief-based likelihood score. Despite using strictly less information, \emph{Pinductor} matches the performance and sample efficiency of LLM-based POMDP learning methods that assume privileged access to the hidden state, while significantly surpassing the sample efficiency of tabular POMDP baselines. Further results show that performance scales with LLM capability and degrades gracefully as semantic information about the environment is withheld. Together, these results position language-model priors as a practical tool for sample-efficient world-model learning under partial observability, and a step toward generalist agents in real-world environments. Code is available at https://github.com/atomresearch/pinductor.


source_id: 2605.13684
source: arxiv
title: Scale-Sensitive Shattering: Learnability and Evaluability at Optimal Scale
authors: ["Shashaank Aiyer","Yishay Mansour","Shay Moran","Han Shao","Tom Waknine"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13684v1
tags: ["robustness to domain shifts","model robustness","PAC-Bayes bounds"]
org: University of Maryland
fetched_at: 2026-05-15T01:47:42.110Z
classified_at: 2026-05-15T21:49:57.424Z
abstract:
We study the optimal scale at which real-valued function classes exhibit uniform convergence and learnability. Our main result establishes a scale-sensitive generalization of the fundamental theorem of PAC learning: for every bounded real-valued class and every $γ>0$, uniform convergence at scale $γ$, agnostic learnability at scale $γ/2$, and finiteness of the fat-shattering dimension at every scale $γ'>γ$ are equivalent. This resolves a question by Anthony and Bartlett (Cambridge Univ. Press 1999) on the precise scales governing learnability, refuting a conjecture attributed there to Phil Long that a multiplicative 2-factor gap is unavoidable, and improves the upper bounds of Bartlett and Long (JCSS 1998), which incur such a loss. The key technical ingredient is a direct bound on empirical $\ell_\infty$ covering numbers, avoiding the standard detour through packing numbers. As a consequence, we obtain sharp asymptotic metric-entropy bounds in terms of the fat-shattering scale $γ$: an $O(\log^2 n)$ bound holds already at scale $γ/2$, while an $O(\log n)$ bound holds at scale $2γ$. We further show that the $O(\log^2 n)$ bound is sometimes tight. These results resolve open questions by Alon et al. (JACM 1997) and Rudelson and Vershynin (Ann. of Math. 2006). As an application, we establish a sharp dichotomy for bounded integral probability metrics: every such IPM is either estimable or cannot be weakly evaluated within any multiplicative factor $c<3$, while $3$-weak evaluability always holds, resolving an open question from Aiyer et al. (ICML 2026). We also highlight several open questions on quantitative sample complexity and evaluability.


source_id: 2605.13471
source: arxiv
authors: ["Narek Maloyan","Dmitry Namiot"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13471v1
tags: []
org: Anthropic
fetched_at: 2026-05-15T01:54:24.102Z
classified_at: 2026-05-15T21:51:07.702Z
title:
Sleeper Channels and Provenance Gates: Persistent Prompt Injection in Always-on Autonomous AI Agents

abstract:
Always-on AI agents (OpenClaw, Hermes Agent) run as a single persistent process under the owner's identity, folding messaging, memory, self-authored skills, scheduling, and shell into one authority boundary. This configuration opens what we call \emph{sleeper channels}: an untrusted input to one surface persists as a memory, skill, scheduled job, or filesystem patch, then fires later through a different surface with no attacker present. Two independent axes define the class: persistence substrate and firing-separation. We walk a confused-deputy cron attack end-to-end through OpenClaw at a pinned commit. The defense is tiered (D1, D2, D3), and D2 carries a soundness theorem against seven named deployment invariants. D2 keys on a canonical action-instance digest with one-shot owner attestations, defeating paraphrase laundering, multi-input grant reuse, and replay. A companion artifact ships the gate, a static audit over the vendored source, and a runtime adapter realising five of the ten mediation hooks (H1, H2, H3, H6, H9) around the cron path (42 tests, Node~$\geq{}20$, at \href{https://github.com/maloyan/sleeper-channels}{github.com/maloyan/sleeper-channels}). Empirical evaluation is preregistered as follow-on.


source_id: 2605.13492
source: arxiv
title: Phantom Force: Injecting Adversarial Tactile Perceptions into Embodied Intelligence via EMI
authors: ["Zirui Kong","Youqian Zhang","Sze Yiu Chau"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13492v1
tags: ["agentic misalignment"]
org: Hong Kong Polytechnic University
fetched_at: 2026-05-15T01:55:40.196Z
classified_at: 2026-05-15T21:51:19.876Z
abstract:
Embodied intelligent robots rely on tactile sensors to interact with the physical world safely. While the security of visual perception systems has been studied (e.g., adversarial samples), the integrity of the tactile sensory channel remains unexplored. This work explores a vulnerability in Hall-effect fingertip sensors, showing their susceptibility to intentional Electromagnetic Interference (EMI). We demonstrate that a targeted signal injection can induce strong ``phantom forces'', amplifying perceived force magnitude by over \textbf{9$\times$} and deviating the inferred force direction by \textbf{65$^\circ$}. Such perturbations can paralyze learning-based tactile classification models, seriously affecting robot movement. An attacker could exploit this vulnerability to coerce a robot hand into crushing fragile objects or dropping dangerous payloads.


source_id: 2605.13338
source: arxiv
authors: ["Shuqiang Wang","Wei Cao","Jiaqi Weng","Jialing Tao","Licheng Pan","Hui Xue","Zhixuan Chu"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13338v1
tags: ["agentic misalignment"]
org: Zhejiang University
fetched_at: 2026-05-15T01:57:56.631Z
classified_at: 2026-05-15T21:51:44.788Z
title:
Inducing Overthink: Hierarchical Genetic Algorithm-based DoS Attack on Black-Box Large Language Reasoning Models

abstract:
Large Reasoning Models (LRMs) are increasingly integrated into systems requiring reliable multi-step inference, yet this growing dependence exposes new vulnerabilities related to computational availability. In particular, LRMs exhibit a tendency to "overthink", producing excessively long and redundant reasoning traces, when confronted with incomplete or logically inconsistent inputs. This behavior significantly increases inference latency and energy consumption, forming a potential vector for denial-of-service (DoS) style resource exhaustion. In this work, we investigate this attack surface and propose an automated black-box framework that induces overthinking in LRMs by systematically perturbing the logical structure of input problems. Our method employs a hierarchical genetic algorithm (HGA) operating on structured problem decompositions, and optimizes a composite fitness function designed to maximize both response length and reflective overthinking markers. Across four state-of-the-art reasoning models, the proposed method substantially amplifies output length, achieving up to a 26.1x increase on the MATH benchmark and consistently outperforming benign and manually crafted missing-premise baselines. We further demonstrate strong transferability, showing that adversarial inputs evolved using a small proxy model retain high effectiveness against large commercial LRMs. These findings highlight overthinking as a shared and exploitable vulnerability in modern reasoning systems, underscoring the need for more robust defenses.


source_id: 2605.13411
source: arxiv
title: Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution
authors: ["Xiaozhe Zhang","Chaozhuo Li","Hui Liu","Shaocheng Yan","Bingyu Yan","Qiwei Ye","Haoliang Li"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13411v1
tags: ["post-training","pretraining","midtraining"]
org: City University of Hong Kong
fetched_at: 2026-05-15T01:59:10.698Z
classified_at: 2026-05-15T21:51:57.564Z
abstract:
Large language models remain vulnerable to adversarial prompts that elicit harmful outputs. Existing safety paradigms typically couple red-teaming and post-training in a closed, policy-centric loop, causing attack discovery to suffer from rapid saturation and limiting the exposure of novel failure modes, while leaving defenses inefficient, rigid, and difficult to transfer across victim models. To this end, we propose EvoSafety, an LLM safety framework built around persistent, inspectable, and reusable external structures. For red teaming, EvoSafety equips the attack policy with an adversarial skill library, enabling continued vulnerability probing through simple library expansion after saturation, while supporting the evolution of adversarial vectors. For defense learning, EvoSafety replaces model-specific safety fine-tuning with a lightweight auxiliary defense model augmented with memory retrieval. This enables efficient, transferable, and model-agnostic safety improvements, while allowing robustness to be enhanced solely through memory updates. With a single training procedure, the defense policy can operate in both Steer and Guard modes: the former activates the victim model's intrinsic defense mechanisms, while the latter directly filters harmful inputs. Extensive experiments demonstrate the superiority of EvoSafety: in Guard mode, it achieves a 99.61% defense success rate, outperforming Qwen3Guard-8B by 14.13% with only 37.5% of its parameters, while preserving reasoning performance on benign queries. Warning: This paper contains potentially harmful text.


source_id: 2605.13214
source: arxiv
title: Backdoor Channels Hidden in Latent Space: Cryptographic Undetectability in Modern Neural Networks
authors: ["Marte Eggen","Eirik Reiestad","Kristian Gjøsteen","Inga Strümke"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13214v1
tags: []
org: Norwegian University of Science and Technology
fetched_at: 2026-05-15T02:02:58.190Z
classified_at: 2026-05-15T21:52:34.528Z
abstract:
Recent cryptographic results establish that neural networks can be backdoored such that no efficient algorithm can distinguish them from a clean model. These guarantees, however, have been confined to stylised architectures of limited practical relevance, leaving open whether comparable undetectability extends to modern, end-to-end trained networks. We construct such an attack mechanism for state-of-the-art architectures, closely aligned to the cryptographic notion of undetectability, by identifying backdoor channels as learned latent directions, and show that the question of undetectability reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice. The consequence of this reframing is significant: if exploitable channels within a network's latent space are statistically indistinguishable from naturally learned directions, an attacker need not introduce foreign structure but can instead exploit the geometry the network already possesses. Demonstrating the approach on ResNet and Vision Transformer architectures trained on standard image classification datasets, the attack achieves both consistently high success rates with negligible clean accuracy degradation, and resists a comprehensive suite of post-training defences, none of which neutralise the backdoor without rendering the model unusable. Our results establish that cryptographic backdoors need not be artefacts requiring exotic architectures or artificial constructions, but identifiable as latent properties inherent to the geometry of learned representations.


source_id: 2605.13163
source: arxiv
title: LoREnc: Low-Rank Encryption for Securing Foundation Models and LoRA Adapters
authors: ["Beomjin Ahn","Jungmin Kwon","Chanyong Jung","Jaewook Chung"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13163v1
tags: ["robustness to domain shifts","model robustness"]
org: University of Michigan
fetched_at: 2026-05-15T02:06:28.558Z
classified_at: 2026-05-15T21:53:10.285Z
abstract:
Foundation models and low-rank adapters enable efficient on-device generative AI but raise risks such as intellectual property leakage and model recovery attacks. Existing defenses are often impractical because they require retraining or access to the original dataset. We propose LoREnc, a training-free framework that secures both FMs and adapters via spectral truncation and compensation. LoREnc suppresses dominant low-rank components of FM weights, compensates for the missing information in authorized adapters, and further applies orthogonal reparameterization to obscure structural fingerprints of the protected adapter. Unauthorized users produce structurally collapsed outputs, while authorized users recover exact performance. Experiments demonstrate that LoREnc provides strong protection against model recovery with under 1% computational overhead.


source_id: 2605.13115
source: arxiv
authors: ["Ziyang You","Liling Zheng","Xiaoke Yang","Xuxing Lu"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13115v1
tags: ["agentic misalignment"]
org: Fujian University of Technology
fetched_at: 2026-05-15T02:09:54.290Z
classified_at: 2026-05-15T21:53:48.843Z
title:
DiffusionHijack: Supply-Chain PRNG Backdoor Attack on Diffusion Models and Quantum Random Number Defense

abstract:
Diffusion models depend on pseudo-random number generators (PRNGs) for latent noise sampling. We present DiffusionHijack, a supply-chain backdoor attack that hijacks the PRNG to deterministically control generated images. A malicious PRNG, injected via compromised packages, forces pixel-perfect reproduction of attacker-chosen content (SSIM = 1.00, N = 100 trials) on Stable Diffusion v1.4, v1.5, and SDXL -- without modifying model weights. The attack is inherently undetectable by existing model auditing and content moderation mechanisms, as it operates entirely outside the neural network computation graph. The attack remains effective under stochastic sampling (eta > 0), bypasses CLIP-based safety checkers (98-100% success), and operates independently of the user's prompt. As a countermeasure, we replace the PRNG with a quantum random number generator (QRNG), which provides information-theoretic unpredictability. Across N = 100 prompt-model combinations, QRNG defense completely neutralizes the attack, reducing output similarity to random baseline levels (SSIM < 0.20 for SD 1.x models, < 0.45 for SDXL). This work exposes a previously overlooked supply-chain vulnerability and offers a hardware-level fundamental mitigation for generative AI systems.


source_id: 2605.13044
source: arxiv
title: No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills
authors: ["Ying Li","Hongbo Wen","Yanju Chen","Hanzhi Liu","Yuan Tian","Yu Feng"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13044v1
tags: ["agent foundations"]
org: University of California, Los Angeles
fetched_at: 2026-05-15T02:11:18.612Z
classified_at: 2026-05-15T21:53:59.831Z
abstract:
LLM-powered agents can silently delete documents, leak credentials, or transfer funds on a routine user request, not because the agent was attacked, but because the skill it invoked broke its own declared safety rules. We call these specification violations: benign inputs cause a skill to breach the natural-language guardrails in its own specification, typically because the guardrail's semantics are undefined for autonomous execution, or because the implementation silently ignores the documented constraint. These violations are invisible to static analyzers, traditional fuzzers, and prompt-injection defenses alike, yet they undermine the very contract a user trusts when installing a skill. We present Sefz, a goal-directed semantic fuzzing framework that automatically discovers specification violations in agent skills. Sefz translates each guardrail into a reachability goal over an annotated execution trace, reducing violation checking to a deterministic graph query. An LLM-based mutator generates benign inputs whose traces progressively approach the violation patterns, guided by a multi-armed bandit that uses goal-proximity as its reward signal. On 402 real-world skills from the largest public agent-skill marketplace, Sefz finds specification violations in 120 (29.9%), including 26 previously unknown exploitable guardrail violations in deployed skills. Six recurring specification pitfalls explain the bulk of the failures, suggesting concrete principles for safer skill design.


source_id: 2605.13095
source: arxiv
title: Watermarking Should Be Treated as a Monitoring Primitive
authors: ["Toluwani Aremu","Nils Lukas","Jie Zhang"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13095v1
tags: ["post-training","watermarking"]
org: MBZUAI
fetched_at: 2026-05-15T02:13:33.295Z
classified_at: 2026-05-15T21:54:22.266Z
abstract:
Watermarking is widely proposed for provenance, attribution, and safety monitoring in generative models, yet is typically evaluated only under adversaries who attempt to evade detection or induce false positives at the level of individual samples. We argue that watermarking should be treated as a monitoring primitive, and that internal monitoring is unavoidable given per-entity attribution keys and messages, as well as detector access. We introduce an observer-based threat model in which observers can aggregate watermark signals across outputs to infer entity-level information, showing that even zero-bit watermarking enables attribution under multi-key settings. We further show that external monitoring can emerge over time from persistent, key-dependent statistical structure, although this depends on watermark design and may be mitigated by distribution-preserving or undetectable schemes. Our findings reveal a fundamental dual-use tension between attribution and monitoring, motivating evaluation of watermarking beyond per-sample robustness to account for aggregation and observer-based capabilities.


source_id: 2605.12990
source: arxiv
authors: ["Muyan Shen","Yu Qin"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12990v1
tags: ["post-training","pretraining","midtraining"]
org: University of Chinese Academy of Sciences
fetched_at: 2026-05-15T02:14:45.942Z
classified_at: 2026-05-15T21:54:33.542Z
title:
Insecure Despite Proven Updated: Extracting the Root VCEK Seed on EPYC Milan via a Software-Only Attack

abstract:
In the official whitepaper of Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP), AMD explicitly emphasizes the capability to prevent Trusted Computing Base (TCB) rollback attacks. Cryptographically, this is realized by signing attestation reports with the Versioned Chip Endorsement Key (VCEK), which is derived by incorporating the TCB version into the hardware root seed. In this architecture, safeguarding the hardware root seed is the ultimate line of defense. However, our research reveals that this protection is insufficient on EPYC Milan by presenting a software-only exploit. Specifically, we firstly introduce MilanLaunchy attack, an exploit that achieves code execution on the AMD secure processor. Building on this foundation, we develop the BadFuse attack, which extracts the hardware root seed by exploiting a lack of write restrictions in the fuse controller. This end-to-end attack chain enables an adversary to forge valid attestation reports for any firmware version, thereby effectively undermining the security model of SEV-SNP.


source_id: 2605.12869
source: arxiv
title: Quantifying LLM Safety Degradation Under Repeated Attacks Using Survival Analysis
authors: ["Zvi Topol"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12869v1
tags: ["post-training","pretraining","midtraining"]
org: MuyVentive LLC
fetched_at: 2026-05-15T02:18:28.418Z
classified_at: 2026-05-15T21:54:55.145Z
abstract:
Large language models (LLMs) are increasingly deployed in a wide range of applications, yet remain vulnerable to adversarial jailbreak attacks that circumvent their safety guardrails. Existing evaluation frameworks typically report binary success/failure metrics, failing to capture the temporal dynamics of how attacks succeed under persistent adversarial pressure. This preliminary work proposes a novel evaluation framework that applies survival analysis techniques to characterize LLM jailbreak vuln`erability. Our approach models the time-to-jailbreak as a survival outcome, enabling estimation of hazard functions, survival curves, and risk factors associated with successful attacks. We evaluate three LLMs against a subset of prompts from the HarmBench dataset spanning three attack categories. Our analysis reveals that models exhibit distinct vulnerability profiles: while one model demonstrates rapid degradation under iterative attacks, the two other models show consistent moderate vulnerability. Our framework provides actionable insights for model and LLM application developers and establishes survival analysis as a rigorous methodology for LLM safety evaluation.


source_id: 2605.12875
source: arxiv
authors: ["Wenhui He","Yue Li","Bang Fu","Huan Xing","Xing Fan","ZeHua Zhang","Baoning Niu"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12875v1
tags: ["agentic misalignment","deception"]
org: Anthropic
fetched_at: 2026-05-15T02:19:40.678Z
classified_at: 2026-05-15T21:55:05.976Z
title:
Do Skill Descriptions Tell the Truth? Detecting Undisclosed Security Behaviors in Code-Backed LLM Skills

abstract:
Programmatic skills in LLM ecosystems consist of a natural-language description and executable implementation files. Users and LLMs rely on the description to understand the skill's scope. However, the implementation may perform security-relevant operations, such as credential access, network communication, or command execution, that the description does not state. We study this description--implementation inconsistency by asking whether the implementation stays within the security-relevant scope declared in the description. We manually analyze 920 real-world programmatic skills and construct an 11-category security property taxonomy. Based on this taxonomy, we build SKILLSCOPE, which constructs source-level security property graphs (SPGs) from implementations and performs LLM-assisted consistency checking. SPG nodes retain source-level code patterns rather than abstract taxonomy labels, preserving fine-grained evidence for checking. On 4,556 programmatic skills with double-blind human review, SKILLSCOPE achieves a precision of 84.8\% and a recall of 96.5\% for identifying inconsistency. Confirmed inconsistency affects 9.4\% of skills, while cases of coarser description, in which implementation details remain within the declared scope, account for 24.3\%. Ablation experiments confirm that both the SPG and the taxonomy contribute: removing the taxonomy reduces precision from 87.8\% to 72.3\%, while removing the SPG reduces recall from 94.7\% to 79.0\%.


source_id: 2605.12863
source: arxiv
title: Language-Based Agent Control
authors: ["Timothy Zhou","Loris D'Antoni","Nadia Polikarpova"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12863v1
tags: ["agent foundations","agentic misalignment","control theory"]
org: University of California-San Diego
fetched_at: 2026-05-15T02:22:08.634Z
classified_at: 2026-05-15T21:55:27.657Z
abstract:
This paper introduces language-based agent control (LBAC), a new programming model for agentic applications that brings techniques from programming languages and language-based security to the problem of agent control. In conventional programming, combinations of static typing and runtime enforcement have long been used to guarantee that well-typed programs satisfy user-specified policies, including policies for access control, information flow, data provenance, and more. The key idea behind LBAC is to extend these guarantees to agentic applications by requiring agents to generate programs that are themselves well typed in the context of the surrounding scaffolding code. Unsafe programs are rejected by the type-checker before execution, allowing policies to apply uniformly across the entire application, including both agent-generated behavior and developer-written scaffolding. At the same time, LBAC preserves substantial expressiveness: agents may perform arbitrary side-effect-free computation and recursively invoke subagents, which retain full tool access subject to the same -- or potentially more restrictive -- policies. We demonstrate LBAC with three case studies: I/O sandboxing via filesystem capabilities, data provenance, and information-flow control.


source_id: 2605.12850
source: arxiv
title: Persona-Model Collapse in Emergent Misalignment
authors: ["Davi Bastos Costa","Renato Vicente"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12850v1
tags: ["agentic misalignment"]
org: University of São Paulo
fetched_at: 2026-05-15T02:24:25.062Z
classified_at: 2026-05-15T21:55:49.573Z
abstract:
Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior on unrelated prompts, a phenomenon known as emergent misalignment. We propose that emergent misalignment involves persona-model collapse: deterioration of the model's internal capacity to simulate, differentiate, and maintain consistent characters. We test this hypothesis behaviorally using two metrics: moral susceptibility (S) and moral robustness (R), computed from the across- and within-persona variability of models' Moral Foundations Questionnaire responses under persona role-play. These metrics formalize the model's ability to differentiate characters (S) and its consistency when simulating a given one (R). We evaluate four frontier models (DeepSeek-V3.1, GPT-4.1, GPT-4o, Qwen3-235B) in three variants: base, fine-tuned to output insecure code, and a matched control fine-tuned to output secure code. Across the four models, insecure fine-tuning produces an average $55\%$ increase in S, pushing all four insecure variants beyond the band observed across 13 frontier models benchmarked in prior work -- with GPT-4o reaching more than twice the band's upper end -- signaling dysregulated differentiation. It also causes an average $65\%$ decrease in R, equivalent to a $304\%$ increase in 1/R. By contrast, the matched secure control preserves S near the base and induces only a partial R loss, showing that these effects are largely misalignment-specific. Complementing these metric shifts, insecure variants' unconditioned responses converge toward saturation near the scale ceiling, departing markedly from both base models' structured responses and those elicited when base models role-play toxic personas. Taken together, these metrics provide a sensitive diagnostic for emergent misalignment and serve as behavioral evidence that it involves persona-model collapse.


source_id: 2605.13639
source: arxiv
title: Achieving $ε^{-2}$ Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions
authors: ["Ishaq Hamza","Zaiwei Chen"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13639v1
tags: ["robustness to domain shifts","model robustness","agent foundations"]
org: Stanford University
fetched_at: 2026-05-15T02:25:37.192Z
classified_at: 2026-05-15T21:56:04.156Z
abstract:
In this paper, we establish last-iterate convergence rates for off-policy actor--critic methods in reinforcement learning. In particular, under a single-loop, single-timescale implementation and a broad class of policy updates, including approximate policy iteration and natural policy gradient methods, we prove the first $\tilde{\mathcal{O}}(ε^{-2})$ sample complexity guarantee for finding an $ε$-optimal policy under minimal assumptions, namely, the existence of a policy that induces an irreducible Markov chain. This stands in stark contrast to the existing literature, where an $\tilde{\mathcal{O}}(ε^{-2})$ sample complexity is achieved only through nested-loop updates and/or under strong, algorithm-dependent assumptions on the policies, such as uniform mixing and uniform exploration. Technically, to address the challenges posed by the coupled update equations arising from the single-loop implementation, as well as the potentially unbounded iterates induced by off-policy learning, our analysis is based on a coupled Lyapunov drift framework. Specifically, we establish a geometric convergence rate for the actor and an $\tilde{\mathcal{O}}(1/T)$ convergence rate for the critic, and combine the two Lyapunov drift inequalities through a cross-domination property. We believe this analytical framework is of independent interest and may be applicable to other coupled iterative algorithms with unbounded


source_id: 2605.13681
source: arxiv
title: Sampling from Flow Language Models via Marginal-Conditioned Bridges
authors: ["Iskander Azangulov","Leo Zhang"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13681v1
tags: []
org: University of Oxford
fetched_at: 2026-05-15T02:27:47.937Z
classified_at: 2026-05-15T21:56:25.382Z
abstract:
Flow Language Models (FLMs) are a recently introduced class of language models which adapt continuous flow matching for one-hot encoded token sequences. Their denoisers have a special structure absent from generic continuous diffusion models: each block of the denoising mean is a posterior marginal distribution over the clean token at that position. Standard DDPM-style samplers collapse these marginals to a single conditional-mean endpoint and bridge toward this simplex-valued point, which is generally not a valid one-hot sequence. We argue that the natural sampler for an FLM is instead posterior-predictive. At each reverse step, we sample a clean one-hot endpoint from the factorized posterior defined by the FLM token marginals, and then sample the next continuous state from the analytic Ornstein--Uhlenbeck bridge conditioned on that endpoint. The method is training-free, uses the same model evaluations as standard sampling, and gives a principled interface for token-level decoding controls such as temperature scaling and nucleus truncation. We show that, under exact posterior marginals, the endpoint approximation error is exactly the conditional multi-information among token positions. The induced one-step bridge kernel preserves all token-wise posterior-predictive marginals and loses only the residual cross-position dependence. Finally, we prove a Girsanov path-space comparison showing that the marginal-conditioned bridge has a no-larger denoising-error term than the frozen conditional-mean bridge, with strict improvement whenever intermediate coordinate-wise bridge observations reveal additional information about the clean token. Experiments with FLMs show that the sampler improves the quality--diversity tradeoff. Code is available at: github.com/imbirik/mcb.


source_id: 2605.13589
source: arxiv
title: Causal Learning with the Invariance Principle
authors: ["Francesco Montagna","Francesco Locatello"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13589v1
tags: ["agent foundations","robustness to domain shifts"]
org: IST Austria
fetched_at: 2026-05-15T02:30:20.898Z
classified_at: 2026-05-15T21:56:52.059Z
abstract:
Causal discovery, the problem of inferring the direction of causality, is generally ill-posed. We use the language of structural causal models (SCM) to show that assuming that the causal relations are acyclic and invariant across multiple environments (e.g., the way minimum wage affects employment rate is stable across different geographical regions), \textit{only} two auxiliary environments are sufficient to infer the causal graph for arbitrary nonlinear mechanisms. Moreover, we demonstrate that this implies identifiability of the SCM functional mechanisms: as a corollary, we show that \textit{two} auxiliary environments are sufficient to guarantee correct counterfactual inference. We empirically support our theoretical results on synthetic data.


source_id: 2605.13612
source: arxiv
title: Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
authors: ["Yatin Dandi","Matteo Vilucchio","Luca Arnaboldi","Hugo Tabanelli","Florent Krzakala"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13612v1
tags: ["agent foundations","model robustness","robustness to domain shifts"]
org: École Polytechnique Fédérale de Lausanne (EPFL)
fetched_at: 2026-05-15T02:31:40.797Z
classified_at: 2026-05-15T21:57:04.190Z
abstract:
Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training in which hierarchical feature learning becomes an explicit iterative spectral procedure. In this limit, the dynamics at each layer decouple: given the current representation, the next layer selects directions with maximal accessible low-degree correlation to the label. This yields a tractable surrogate mechanism for deep learning, together with a natural kernel-space interpretation. Neural LoFi provides a mathematically explicit framework for studying multi-layer feature learning beyond the lazy regime. It predicts how representations are selected layer by layer, explains how emergence of concepts arises with given sample complexity,and gives a concrete mechanism by which depth progressively constructs new features from old ones through low-degree compositionality. We complement the theory with mechanistic experiments on fully connected and convolutional architectures, showing that Neural LoFi improves over lazy random-feature baselines, recovers meaningful structured filters, and predicts representations aligned with early gradient-descent feature discovery with real datasets.


source_id: 2605.13448
source: arxiv
title: On the Limits of Latent Reuse in Diffusion Models
authors: ["Yifeng Yu","Lu Yu"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13448v1
tags: ["robustness to domain shifts","model robustness"]
org: Tsinghua University
fetched_at: 2026-05-15T02:34:03.276Z
classified_at: 2026-05-15T21:57:30.748Z
abstract:
Diffusion models are often trained in low-dimensional latent spaces, which are then reused for related but shifted datasets. In this work, we study when such latent reuse remains reliable under distribution shift. We consider a source-target setting in which both datasets are approximately low-dimensional but may lie near different subspaces. We show that freezing and reusing a source latent space induces a target-domain score error governed by two quantities: the principal-angle misalignment between the source and target subspaces, and the target ambient noise amplified by the diffusion time scale. Motivated by these limits, we further study mixed source-target training and characterize how the required shared latent dimension depends on the relative geometry of the two distributions. Our results provide theoretical guidance on when latent reuse is reliable and when learning a shared representation may be necessary.


source_id: 2605.13284
source: arxiv
title: Learning Perturbations to Extrapolate Your LLM
authors: ["Zetai Cen","Chenfei Gu","Jin Zhu","Ting Li","Yunxiao Chen","Chengchun Shi"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13284v1
tags: ["post-training","pretraining","midtraining"]
org: University of Bristol
fetched_at: 2026-05-15T02:36:15.480Z
classified_at: 2026-05-15T21:57:52.341Z
abstract:
Recent advancements in large language models demonstrate that injecting perturbations can substantially enhance extrapolation performance. However, current approaches often rely on discrete perturbations with fixed designs, which limits their flexibility. In this work, we propose a framework where token prefixes are perturbed by a learnable transformation of a continuous latent vector within an embedding space. To overcome the challenge of an intractable marginal likelihood, we derive unbiased estimating equations for model parameters and optimize them via stochastic gradient descent. We establish the statistical properties of the resulting estimator in over-parameterized regimes. Empirical evaluations on both synthetic and real-world datasets demonstrate that our proposal yields significant gains in out-of-domain settings over a range of state-of-the-art baseline methods.


source_id: 2605.13287
source: arxiv
title: Delightful Exploration
authors: ["Ian Osband"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13287v1
tags: ["agent foundations","robustness to domain shifts","model robustness"]
org: Google DeepMind
fetched_at: 2026-05-15T02:37:23.706Z
classified_at: 2026-05-15T21:58:04.807Z
abstract:
Most exploration algorithms search broadly until uncertainty is resolved. When the action space is too large to resolve within budget, practitioners default to $\varepsilon$-greedy, which bounds disruption but spends its override blindly. We introduce \textit{Delight-gated exploration} (DE), a host--override rule that spends exploratory actions only when their prospective delight (expected improvement times surprisal) exceeds a gate price. This practical heuristic recovers a classical result: Pandora's reservation-value rule for costly search, with surprisal setting the effective inspection cost. Resolved arms exit the gate, fresh arms shut off above a prior-determined threshold, and selected linear-bandit overrides consume finite information budget. Across Bernoulli bandits, linear bandits, and tabular MDPs, the same hyperparameters transfer without retuning, and DE shows much weaker regret growth than Thompson Sampling and $\varepsilon$-greedy in the tested unresolved regimes. Delight improves acting for the same reason it improves learning: it prices scarce resources by the product of upside and surprisal.


source_id: 2605.13188
source: arxiv
title: LLMs as Implicit Imputers: Uncertainty Should Scale with Missing Information
authors: ["Stef van Buuren"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13188v1
tags: ["model robustness","uncertainty quantification"]
org: TNO
fetched_at: 2026-05-15T02:42:15.784Z
classified_at: 2026-05-15T21:58:53.525Z
abstract:
Large language models (LLMs) are increasingly deployed in settings where the available context is incomplete or degraded. We argue that an LLM generating answers under incomplete context can be viewed as an implicit imputer, and evaluated against a criterion from the multiple imputation (MI) literature: uncertainty should scale with the amount of missing information. We assess this criterion on SQuAD, using a controlled framework in which context availability is varied across five levels. We evaluate two answer-level uncertainty measures that can be estimated from repeated sampling: sampling-based confidence (empirical mode frequency) and response entropy. Confidence fails to reflect increasing missingness: it remains high even as accuracy collapses. Entropy, by contrast, increases with context removal, consistent with the MI analogy, and explains substantially more variance in accuracy than confidence across all evidence levels (quadratic $R^2$ gap up to 0.057). We further introduce a black-box diagnostic $ρ_R(α)$ that estimates the proportion of baseline uncertainty resolved by context level $α$, requiring only repeated sampling with and without context. These results suggest that entropy is a more responsive black-box uncertainty measure than confidence under incomplete context.


source_id: 2605.13146
source: arxiv
title: On Hallucinations in Inverse Problems: Fundamental Limits and Provable Assessment Methods
authors: ["David Iagaru","Nina M. Gottschling","Anders C. Hansen","Josselin Garnier"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13146v1
tags: ["agentic misalignment","hallucinations"]
org: Deutsches Zentrum für Luft- und Raumfahrt (DLR)
fetched_at: 2026-05-15T02:48:43.786Z
classified_at: 2026-05-15T22:00:01.688Z
abstract:
Artificial intelligence (AI) has transformed imaging inverse problems, from medical diagnostics to Earth observation. Yet deep neural networks can produce hallucinations, realistic-looking but incorrect details, undermining their reliability, especially when ground truth data is unavailable. We develop a theoretical framework showing that such hallucinations are not merely artifacts of particular models, but can arise from the ill-posed nature of the inverse problem itself. We derive necessary and sufficient conditions for hallucinations, together with computable bounds on their magnitude that depend only on the forward model. Building on this theory, we introduce algorithms to: (1) estimate the minimum hallucination magnitude achievable by any reconstruction model for a given input; (2) assess the faithfulness of reconstructed details by a given reconstruction model. Experiments across three imaging tasks demonstrate that our approach applies broadly, including to modern generative models, and provides a principled way to quantify and evaluate AI hallucinations.


source_id: 2605.12908
source: arxiv
title: The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
authors: ["Ryoya Awano","Taiji Suzuki"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12908v1
tags: []
org: University of Tokyo
fetched_at: 2026-05-15T02:54:49.937Z
classified_at: 2026-05-15T22:00:54.615Z
abstract:
Weak-to-strong (W2S) generalization, in which a strong model is fine-tuned on outputs of a weaker, task-specialized model, has been proposed as an approach to aligning superhuman AI systems. Existing theoretical analyses either fix the student's representations or operate in restricted settings. Whether multi-step SGD can succeed in feature learning while preserving diverse pre-trained capabilities remains open. We study W2S in the setting of reward-model learning with two-layer neural networks. The strong model has pre-trained representations organized into low-dimensional subspaces $V_k$, and is fine-tuned under the supervision of a weak model specialized on task $κ$. We prove that the strong model efficiently learns task $κ$, eliciting its pre-trained knowledge while retaining general capabilities. This establishes W2S generalization in the feature-learning regime, in the sense that the strong model acquires the target feature direction through W2S training, rather than having it given a priori. Moreover, W2S preserves pre-trained off-target features, whereas standard supervised fine-tuning causes catastrophic forgetting when off-target feature directions are correlated with the target's. Numerical experiments on synthetic data confirm our theoretical results.


source_id: 2605.12947
source: arxiv
title: When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems
authors: ["Young Hyun Cho","Will Wei Sun"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.12947v1
tags: ["post-training","pretraining","midtraining"]
org: Purdue University
fetched_at: 2026-05-15T02:55:51.982Z
classified_at: 2026-05-15T22:01:05.454Z
abstract:
LLM-enabled AI workflows increasingly produce outputs through iterative generate-evaluate-revise loops. Each iteration can improve the candidate, but it also creates a release decision: when to stop and output the current result? This raises a statistical challenge because deployment-time evaluator scores are adaptively generated and repeatedly monitored, yet the likelihood models or exchangeability assumptions typically used for calibration are unavailable. We propose an always-valid release wrapper for existing generator-evaluator pipelines. The wrapper builds a hard-negative reference pool of high-scoring failures, calibrates deployment-time evaluator scores against this pool, and accumulates the resulting evidence with an e-process. This separates two roles: the reference pool turns black-box scores into conservative evidence, while the e-process provides validity under optional stopping. In theory, we show that a conservative reference pool yields finite-sample control of the probability of releasing on infeasible tasks, that is, tasks for which the given workflow is not capable of producing a reliable solution. We also characterize conditions under which the same conservative rule still achieves nontrivial release on feasible tasks. In an MBPP+ coding-agent case study, the wrapper reduces premature incorrect release relative to baseline stopping rules while still releasing on tasks for which the workflow repeatedly accumulates moderate supporting evidence.


source_id: 2605.13839
source: arxiv
title: Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights
authors: ["Wenrui Bao","Huan Wang","Jian Wang","Zhangyang Wang","Kai Wang","Yuzhang Shang"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13839v1
tags: ["agent foundations","robustness to domain shifts","model robustness"]
org: University of Central Florida
fetched_at: 2026-05-15T02:57:00.708Z
classified_at: 2026-05-15T22:01:16.767Z
abstract:
Multi-agent LLM systems usually collaborate by exchanging natural-language messages. This interface is simple and interpretable, but it forces each sender's intermediate computation to be serialized into tokens and then reprocessed by the receiver, thereby increasing the generated-token cost, prefill overhead, and KV-cache memory. We study an alternative communication interface: instead of appending a sender's message to the receiver's context, compile the sender's hidden states into a transient, receiver-specific weight perturbation. We introduce TFlow (Thought Flow), a weight-space communication framework for a known and fixed receiver architecture. For each query, frozen role-prompted sender agents process the input, and a learned parameter generator maps their internal activations into low-rank LoRA perturbations targeting the receiver's modules. These perturbations are fused and applied only during the receiver's generation, enabling instance-level adaptation without permanently changing the model or enlarging the receiver's text context. With three Qwen3-4B agents, TFlow improves over a standalone receiver by up to 8.5 accuracy points across five benchmarks while reducing processed tokens by up to 32.69%. Compared with a text-based three-agent baseline, it reduces total processed tokens by up to 83.27% and the wall-clock inference time by up to 4.6$\times$, while maintaining competitive accuracy on four of five benchmarks. These results suggest that transient low-rank weight perturbations can serve as an executable communication medium for efficient multi-agent LLM collaboration.


source_id: 2605.13641
source: arxiv
title: Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13641v1
tags: ["reward specification","reinforcement learning","reinforcement learning from human feedback"]
org: Meituan
fetched_at: 2026-05-15T03:02:00.862Z
classified_at: 2026-05-15T22:02:02.404Z
authors:
["Yang Bai","Kaiyuan Liu","Ziyuan Zhuang","Jiahong Zhou","Rongxiang Weng","Xin Chen","Jingang Wang","Xunliang Cai"]

abstract:
Complex reinforcement learning environments frequently employ multi-task and mixed-reward formulations. In these settings, heterogeneous reward distributions and correlated reward dimensions often destabilize the construction of scalar advantages. To address these challenges, we propose Reward-Decorrelated Policy Optimization (RDPO), a reward-processing method designed to explicitly target both failure modes. RDPO first utilizes Magnitude-Aware Quantile normalization to stabilize prompt-level advantage allocation across binary, fractional, and continuous rewards. It then applies Mahalanobis whitening within each active reward subspace to mitigate correlation redundancy prior to aggregation. When applied during the post-training of LongCat-Flash, RDPO enhances instruction following, writing quality, and robustness to hard prompts while remaining broadly competitive on reasoning and coding evaluations.


source_id: 2605.13624
source: arxiv
title: Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction
authors: ["Takumi Goto","Yusuke Sakai","Taro Watanabe"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13624v1
tags: ["post-training","pretraining"]
org: Nara Institute of Science and Technology (NAIST)
fetched_at: 2026-05-15T03:05:26.886Z
classified_at: 2026-05-15T22:02:23.781Z
abstract:
Grammatical error correction using large language models often suffers from the over-correction issue. To mitigate this, we propose a training-free inference method that performs edit-level majority voting over multiple candidates generated by a single model, without requiring model modifications or additional training. Across nine benchmarks covering English, Czech, German, Ukrainian, Korean, Hindi, and Romanian, the proposed method outperforms both greedy and MBR decoding in most cases. Moreover, it yields stable correction quality regardless of the instruction prompts used. We release two repository supporting GEC datasets loading and LLM inference.


source_id: 2605.13595
source: arxiv
title: Inducing Artificial Uncertainty in Language Models
authors: ["Sophia Hager","Simon Zeng","Nicholas Andrews"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13595v1
tags: ["evaluation","training","interpretability"]
org: Johns Hopkins University
fetched_at: 2026-05-15T03:06:27.675Z
classified_at: 2026-05-15T22:02:34.100Z
abstract:
In safety-critical applications, language models should be able to characterize their uncertainty with meaningful probabilities. Many uncertainty quantification approaches require supervised data; however, finding suitable unseen challenging data is increasingly difficult for large language models trained on vast amounts of scraped data. If the model is consistently (and correctly) confident in its predictions, the uncertainty quantification method may consistently overestimate confidence on new and unfamiliar data. Finding data which exhibits enough uncertainty to train supervised uncertainty quantification methods for high-performance models may therefore be challenging, and will increase in difficulty as LLMs saturate datasets. To address this issue, we first introduce the problem of inducing artificial uncertainty in language models, then investigate methods of inducing artificial uncertainty on trivially easy data in the absence of challenging data at training time. We use probes trained to recognize artificial uncertainty on the original model, and find that these probes trained on artificial uncertainty outperform probes trained without artificial uncertainty in recognizing real uncertainty, achieving notably higher calibration on hard data with minimal loss of performance on easy data.


source_id: 2605.13542
source: arxiv
title: RealICU: Do LLM Agents Understand Long-Context ICU Data? A Benchmark Beyond Behavior Imitation
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13542v1
tags: ["evaluation","model infrastructure","monitoring"]
org: Cherise
fetched_at: 2026-05-15T03:06:54.786Z
classified_at: 2026-05-15T22:02:45.171Z
authors:
["Chengzhi Shen","Weixiang Shen","Tobias Susetzky","Chen","Chen","Jun Li","Yuyuan Liu","Xuepeng Zhang","Zhenyu Gong","Daniel Rueckert","Jiazhen Pan"]

abstract:
Intensive care units (ICU) generate long, dense and evolving streams of clinical information, where physicians must repeatedly reassess patient states under time pressure, underscoring a clear need for reliable AI decision support. Existing ICU benchmarks typically treat historical clinician actions as ground truth. However, these actions are made under incomplete information and limited temporal context of the underlying patient state, and may therefore be suboptimal, making it difficult to assess the true reasoning capabilities of AI systems. We introduce RealICU, a hindsight-annotated benchmark for evaluating large language models (LLMs) under realistic ICU conditions, where labels are created after senior physicians review the full patient trajectory. We formulate four physician-motivated tasks: assess Patient Status, Acute Problems, Recommended Actions, and Red Flag actions that risk unsafe outcomes. We partition each trajectory with 30-min windows and release two datasets: RealICU-Gold with 930-window annotations from 94 MIMIC-IV patients, and RealICU-Scale with 11,862 windows extended by Oracle, a physician-validated LLM hindsight labeler. Existing LLMs including memory-augmented ones performed poorly on RealICU, exposing two failure modes: a recall-safety tradeoff for clinical recommendations, and an anchoring bias to early interpretations of the patient. We further introduce ICU-Evo to study structured-memory agents that improves long-horizon reasoning but does not fully eliminate safety failures. Together, RealICU provides a clinically grounded testbed for measuring and improving AI sequential decision-support in high-stakes care. Project page: https://chengzhi-leo.github.io/RealICU-Bench/


source_id: 2605.13537
source: arxiv
title: Temper and Tilt Lead to SLOP: Reward Hacking Mitigation with Inference-Time Alignment
authors: ["Ye Wang","Jing Liu","Toshiaki Koike-Akino"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13537v1
tags: ["inference-time controls","alignment"]
org: MIT
fetched_at: 2026-05-15T03:08:04.139Z
classified_at: 2026-05-15T22:02:55.307Z
abstract:
Inference-time alignment techniques offer a lightweight alternative or complement to costly reinforcement learning, while enabling continual adaptation as alignment objectives and reward targets evolve. Existing theoretical analyses justify these methods as approximations to sampling from distributions optimally tilted toward a given reward model. We extend these techniques by introducing reference-model temperature adjustment, which leads to further generalization of inference-time alignment to ensembles of generative reward models combined as a sharpened logarithmic opinion pool (SLOP). To mitigate reward hacking, we propose an algorithm for calibrating SLOP weight parameters and experimentally demonstrate that it improves robustness while preserving alignment performance.


source_id: 2605.13486
source: arxiv
title: R^2-Mem: Reflective Experience for Memory Search
authors: ["Xinyuan Wang","Wenyu Mao","Junkang Wu","Xiang Wang","Xiangnan He"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13486v1
tags: ["model infrastructure","retrieval augmented generation","evaluation"]
org: University of Science and Technology of China
fetched_at: 2026-05-15T03:12:00.034Z
classified_at: 2026-05-15T22:03:28.100Z
abstract:
Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat past error behaviors because they fail to learn from the prior high- and low-quality search trajectories. To address this limitation, we propose R^2-Mem, a reflective experience framework for memory search systems. In the offline stage, a Rubric-guided Evaluator scores low- and high-quality steps in historical trajectories, and a self-Reflection Learner distills the corresponding abstract experience. During the online inference, the retrieved experience will guide future search actions to avoid repeated mistakes and maintain high-quality behaviors. Extensive experiments demonstrate that R^2-Mem consistently improves both effectiveness and efficiency over strong baselines, improving F1 scores by up to 22.6%, while reducing token consumption by 12.9% and search iterations by 20.2%. These results verify that R^2-Mem provides a RL-free and low-cost solution for self-improving LLM agents.


source_id: 2605.13438
source: arxiv
title: Cognifold: Always-On Proactive Memory via Cognitive Folding
authors: ["Suli Wang","Yiqun Duan","Yu Deng","Rundong Zhao","Dai Shi","Xinliang Zhou"]
published_date: 2026-05-13
pdf_url: https://arxiv.org/pdf/2605.13438v1
tags: ["model infrastructure","evaluation","agent foundations"]
org: OpenNorve
fetched_at: 2026-05-15T03:24:07.066Z
classified_at: 2026-05-15T22:05:24.248Z
abstract:
Existing agent memory remains predominantly reactive and retrieval-based, lacking the capacity to autonomously organize experience into persistent cognitive structure. Toward genuinely autonomous agents, we introduce Cognifold, a brain-inspired "always-on" agent memory designed for the next generation of proactive assistants. CogniFold continuously folds fragmented event streams into self-emerging cognitive structures, bootstrapping progressively higher-level cognition from incoming events and accumulated knowledge. We ground this by extending Complementary Learning Systems (CLS) theory from two layers (hippocampus, neocortex) to three, adding a prefrontal intent layer. Emulating the prefrontal cortex as the locus of intentional control and decision-making, CogniFold achieves this through graph-topology self-organization: cognitive structures proactively assemble under the stream, merge when semantically similar, decay when stale, relink through associative recall, and surface intents when concept-cluster density crosses a threshold. We evaluate structural formation using CogEval-Bench, demonstrating that CogniFold uniquely produces memory structures that match cognitive expectations and concept emergence. Furthermore, across 7 broad-coverage benchmarks spanning five cognitive domains, we validate that CogniFold simultaneously performs robustly on conventional memory benchmarks.


source_id: fYF8v2ukZmsNvmkkX
source: lesswrong
title: Convergent Abstraction Hypothesis
authors: ["Jan_Kulveit"]
published_date: 2026-05-15
pdf_url: https://www.lesswrong.com/posts/fYF8v2ukZmsNvmkkX/convergent-abstraction-hypothesis
tags: ["agent foundations","interpretability","theories and proofs"]
org: Alignment Forum
fetched_at: 2026-05-15T03:33:05.385Z
classified_at: 2026-05-15T22:07:00.308Z
abstract:
Convergent abstraction hypothesis posits abstractions are often convergent in the sense of convergent evolution : different cognitive systems converge on the same abstraction, when facing similar selection pressures and learning in similar environments. It is a less ambitious alternative to 'natural abstractions hypotheses' and, in my view, more likely to be true. Convergence may be real, useful, and empirically robust, while still being contingent and fragile under changes in architecture, training pressure, or optimization regime. Convergent evolution To recapitulate some basics : notice that sharks, ichthyosaurs and dolphins look remarkably similar, despite their ancestors being wildly different. This is a case of convergent evolution: the process by which organisms with different origins develop similar features. Both sharks and dolphins needed speed and energy efficiency when moving in an environment governed by the laws of hydrodynamics, and so they converged on a pretty similar body shape. A complementary concept from biology is contingency - convergent features are themselves often contingent on some other feature. For example, the shark morphology depends on there be


source_id: Bh4pBdAQfgoJ7CFoP
source: lesswrong
authors: ["burnssa"]
published_date: 2026-05-14
tags: ["evaluation","alignment","model infrastructure"]
org: Alignment Forum
fetched_at: 2026-05-15T03:34:10.974Z
classified_at: 2026-05-15T22:07:25.496Z
title:
2B scoring model flags out-of-domain misalignment, suggesting specialist judges have potential for audits

abstract:
Some evidence that narrow ‘specialist’ models could be useful as part of deployed model misalignment audits, complementing larger frontier auditing agents and offering potential cost, discrimination and transparency benefits. A Gemma 2B specialist judge trained on Betley et al 2025b (‘Betley’) code examples is able to distinguish between responses provided by insecure-fine-tuned ‘misaligned’ and ‘secure-fine-tuned’ models responding to out-of-domain general safety prompts (‘ICEBERG’ - details below), where prompted Sonnet 4.5 judgments on the same prompts are not discriminative Similar tests on SecurityEval code prompts (out-of-distribution but in-domain) show the specialist judges track alignment drift and improve on Sonnet's paired discrimination (winrate of ~66% vs ~64%), but comparisons are partly confounded by pattern-matching Betley training data attributes The results tentatively indicate promise for extending narrow classifiers now often used in safety contexts (e.g. Anthropic classifiers for pretraining filtering and jailbreak protection ) to deployed model audits, but more work is required to prove specialist judges' value Replication guide here Scalable auditing of

pdf_url:
https://www.lesswrong.com/posts/Bh4pBdAQfgoJ7CFoP/2b-scoring-model-flags-out-of-domain-misalignment-suggesting


source_id: ahu9BCHPFDg9AJF75
source: lesswrong
title: Some observations about NLA explanations
authors: ["loops"]
published_date: 2026-05-15
pdf_url: https://www.lesswrong.com/posts/ahu9BCHPFDg9AJF75/some-observations-about-nla-explanations
tags: ["post-training","pretraining"]
org: Alignment Forum
fetched_at: 2026-05-15T03:34:41.522Z
classified_at: 2026-05-15T22:07:36.096Z
abstract:
I used the Gemma 3 12B activation verbalizer (maps activations to English) and reconstructor (maps English to activations) described in the Natural Language Autoencoders (NLA) paper to generate a bunch of explanations for 20k random tokens from a pretraining dataset (Common Pile derivative) and another 20k random tokens from a chat dataset . I also reconstructed all of the activations from the verbalizations so that I could see what kinds of tokens and explanations have high reconstruction error. I published pretraining token explanations and chat token explanations [1] and the code I used (written with Claude). Here are some thoughts I have about the explanations and some experiments I did with the data. Explanation text qualities Almost all of the Gemma explanations I read follow the same three part format, with each part is being 1-2 sentences. A sentence that describes the kind of document we're in, then a colon, then a description of what the document's about. For chat sessions this usually starts with the word "Structured" (80% for chat sessions vs 7% for non-chat). Two examples (non-chat vs chat): Literary narrative pattern: formal English countryside setting, with a medieva


source_id: tK8vqHDxaRGcysNJQ
source: lesswrong
authors: ["Charlie Griffin"]
published_date: 2026-05-14
tags: ["evaluation","alignment","safety and security"]
org: Alignment Forum
fetched_at: 2026-05-15T03:35:45.424Z
classified_at: 2026-05-15T22:08:01.160Z
title:
The safe-to-dangerous shift is a fundamental problem for eval realism; but also for measuring awareness

abstract:
1) The safe-to-dangerous shift is a fundamental problem for eval realism Suppose we have a capable and potentially scheming model, and before we deploy it, we want some evidence that it won’t do anything catastrophically dangerous once we deploy it. A common approach is to use black-box alignment evaluations. However, alignment evaluations are only reassuring to the extent that the model can't reliably [1] distinguish the deployment distribution from the evaluation distribution, as it is otherwise difficult to rule out the possibility of alignment faking . There are many approaches one could use to try to make evaluations appear more realistic: you can try to create realistic environments (e.g. Petri , WebArena , OSWorld ); use data from past deployments (e.g. OpenAI , SAD ); and spoof tool-call responses (e.g. ToolEmu ). However, the core difference between an alignment evaluation and a real deployment, is that an alignment evaluation must be safe [2] . That is, if you want to safely evaluate an untrusted AI system, you need to limit your AI’s ability to cause harm within your evaluation environment. If you want to deploy your AI system (usefully) you need to give it at least some

pdf_url:
https://www.lesswrong.com/posts/tK8vqHDxaRGcysNJQ/the-safe-to-dangerous-shift-is-a-fundamental-problem-for-1


source_id: 2605.15188
source: arxiv
title: FutureSim: Replaying World Events to Evaluate Adaptive Agents
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15188v1
tags: ["evaluation","safety and security","monitoring"]
org: University of Tübingen
fetched_at: 2026-05-16T00:32:35.502Z
classified_at: 2026-05-16T00:32:35.428Z
authors:
["Shashwat Goel","Nikhil Chandak","Arvindh Arun","Ameya Prabhu","Steffen Staab","Moritz Hardt","Maksym Andriushchenko","Jonas Geiping"]

abstract:
AI agents are being increasingly deployed in dynamic, open-ended environments that require adapting to new information as it arrives. To efficiently measure this capability for realistic use-cases, we propose building grounded simulations that replay real-world events in the order they occurred. We build FutureSim, where agents forecast world events beyond their knowledge cutoff while interacting with a chronological replay of the world: real news articles arriving and questions resolving over the simulated period. We evaluate frontier agents in their native harness, testing their ability to predict world events over a three-month period from January to March 2026. FutureSim reveals a clear separation in their capabilities, with the best agent's accuracy being 25%, and many having worse Brier skill score than making no prediction at all. Through careful ablations, we show how FutureSim offers a realistic setting to study emerging research directions like long-horizon test-time adaptation, search, memory, and reasoning about uncertainty. Overall, we hope our benchmark design paves the way to measure AI progress on open-ended adaptation spanning long time-horizons in the real world.


source_id: 2605.15198
source: arxiv
title: ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both
authors: ["Ziyu Guo","Rain Liu","Xinyan Chen","Pheng-Ann Heng"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15198v1
tags: ["reinforcement learning","training","interpretability"]
org: Meta AI
fetched_at: 2026-05-16T00:34:05.529Z
classified_at: 2026-05-16T00:34:05.494Z
abstract:
Visual reasoning, often interleaved with intermediate visual states, has emerged as a promising direction in the field. A straightforward approach is to directly generate images via unified models during reasoning, but this is computationally expensive and architecturally non-trivial. Recent alternatives include agentic reasoning through code or tool calls, and latent reasoning with learnable hidden embeddings. However, agentic methods incur context-switching latency from external execution, while latent methods lack task generalization and are difficult to train with autoregressive parallelization. To combine their strengths while mitigating their limitations, we propose ATLAS, a framework in which a single discrete 'word', termed as a functional token, serves both as an agentic operation and a latent visual reasoning unit. Each functional token is associated with an internalized visual operation, yet requires no visual supervision and remains a standard token in the tokenizer vocabulary, which can be generated via next-token prediction. This design avoids verbose intermediate visual content generation, while preserving compatibility with the vanilla scalable SFT and RL training, without architectural or methodological modifications. To further address the sparsity of functional tokens during RL, we introduce Latent-Anchored GRPO (LA-GRPO), which stabilizes the training by anchoring functional tokens with a statically weighted auxiliary objective, providing stronger gradient updates. Extensive experiments and analyses demonstrate that ATLAS achieves superior performance on challenging benchmarks while maintaining clear interpretability. We hope ATLAS offers a new paradigm inspiring future visual reasoning research.


source_id: 2605.15179
source: arxiv
authors: ["Ellwil Sharma","Arastu Sharma"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15179v1
tags: ["model infrastructure","training","theories and proofs"]
org: Stanford University
fetched_at: 2026-05-16T00:37:06.532Z
classified_at: 2026-05-16T00:37:06.496Z
title:
Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing

abstract:
Scaling Scientific Machine Learning (SciML) toward universal foundation models is bottlenecked by negative transfer: the simultaneous co-training of disparate partial differential equation (PDE) regimes can induce gradient conflict, unstable optimization, and plasticity loss in dense neural operators. In particular, broadband open-channel fluid dynamics and boundary-dominated porous media flows impose incompatible spectral and geometric demands on a single dense parameter path. We introduce Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport. Shodh-MoE operates on compressed 16^3 physical latents produced by a physics-informed autoencoder with an intra-tokenizer Helmholtz-style velocity parameterization, restricting decoded states to divergence-free velocity manifolds. The model guarantees exact mass conservation, achieving a physically verifiable velocity divergence of ~2.8 x 10^-10 (evaluated post-hoc in FP64) on 128^3 grids. A Top-1 soft-semantic router dynamically assigns localized latent patches to expert subnetworks, enabling specialized parameter paths for distinct physical mechanisms while preserving shared experts for universal symmetries. In a 20,000-step distributed pretraining run over mixed three-dimensional physical tensors, routing telemetry shows autonomous domain bifurcation: held-out validation tokens from the open-channel domain route exclusively to Expert 0, while porous-media tokens route exclusively to Expert 1. The model converges simultaneously across both regimes, achieving latent validation MSEs of 2.46 x 10^-5 and 9.76 x 10^-6, and decoded physical MSEs of 2.48 x 10^-6 and 1.76 x 10^-6. These results support sparse expert routing as a practical architectural mechanism for mitigating multi-physics interference in universal neural operators.


source_id: 2605.15186
source: arxiv
title: VGGT-Edit: Feed-forward Native 3D Scene Editing with Residual Field Prediction
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15186v1
tags: ["model infrastructure","training","theories and proofs"]
org: Peking University
fetched_at: 2026-05-16T00:38:40.133Z
classified_at: 2026-05-16T00:38:40.103Z
authors:
["Kaixin Zhu","Yiwen Tang","Yifan Yang","Renrui Zhang","Bohan Zeng","Ziyu Guo","Ruichuan An","Zhou Liu","Qizhi Chen","Delin Qu","Jaehong Yoon","Wentao Zhang"]

abstract:
High-quality 3D scene reconstruction has recently advanced toward generalizable feed-forward architectures, enabling the generation of complex environments in a single forward pass. However, despite their strong performance in static scene perception, these models remain limited in responding to dynamic human instructions, which restricts their use in interactive applications. Existing editing methods typically rely on a 2D-lifting strategy, where individual views are edited independently and then lifted back into 3D space. This indirect pipeline often leads to blurry textures and inconsistent geometry, as 2D editors lack the spatial awareness required to preserve structure across viewpoints. To address these limitations, we propose VGGT-Edit, a feed-forward framework for text-conditioned native 3D scene editing. VGGT-Edit introduces depth-synchronized text injection to align semantic guidance with the backbone's spatial poses, ensuring stable instruction grounding. This semantic signal is then processed by a residual transformation head, which directly predicts 3D geometric displacements to deform the scene while preserving background stability. To ensure high-fidelity results, we supervise the framework with a multi-term objective function that enforces geometric accuracy and cross-view consistency. We also construct the DeltaScene Dataset, a large-scale dataset generated through an automated pipeline with 3D agreement filtering to ensure ground-truth quality. Experiments show that VGGT-Edit substantially outperforms 2D-lifting baselines, producing sharper object details, stronger multi-view consistency, and near-instant inference speed.


source_id: 2605.15185
source: arxiv
title: Quantitative Video World Model Evaluation for Geometric-Consistency
authors: ["Jiaxin Wu","Yihao Pi","Yinling Zhang","Yuheng Li","Xueyan Zou"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15185v1
tags: ["evaluation","model infrastructure","theories and proofs"]
org: Adobe Research
fetched_at: 2026-05-16T00:40:05.977Z
classified_at: 2026-05-16T00:40:05.939Z
abstract:
Generative video models are increasingly studied as implicit world models, yet evaluating whether they produce physically plausible 3D structure and motion remains challenging. Most existing video evaluation pipelines rely heavily on human judgment or learned graders, which can be subjective and weakly diagnostic for geometric failures. We introduce PDI-Bench (Perspective Distortion Index), a quantitative framework for auditing geometric coherence in generated videos. Given a generated clip, we obtain object-centric observations via segmentation and point tracking (e.g., SAM 2, MegaSaM, and CoTracker3), lift them to 3D world-space coordinates via monocular reconstruction, and compute a set of projective-geometry residuals capturing three failure dimensions: scale-depth alignment, 3D motion consistency, and 3D structural rigidity. To support systematic evaluation, we build PDI-Dataset, covering diverse scenarios designed to stress these geometric constraints. Across state-of-the-art video generators, PDI reveals consistent geometry-specific failure modes that are not captured by common perceptual metrics, and provides a diagnostic signal for progress toward physically grounded video generation and physical world model. Our code and dataset can be found at https://pdi-bench.github.io/.


source_id: 2605.15177
source: arxiv
title: OpenDeepThink: Parallel Reasoning via Bradley--Terry Aggregation
authors: ["Shang Zhou","Wenhao Chai","Kaiyuan Liu","Huanzhi Mao","Qiuyang Mang","Jingbo Shang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15177v1
tags: ["model infrastructure","evaluation","reinforcement learning"]
org: Stanford University
fetched_at: 2026-05-16T00:41:39.817Z
classified_at: 2026-05-16T00:41:39.776Z
abstract:
Test-time compute scaling is a primary axis for improving LLM reasoning. Existing methods primarily scale depth by extending a single reasoning trace. Scaling breadth by sampling multiple candidates in parallel is straightforward, but introduces a selection bottleneck: choosing the best candidate without a ground-truth verifier, since pointwise LLM judging is noisy and biased. To address this, we introduce OpenDeepThink, a population-based test-time compute framework that selects via pairwise Bradley-Terry comparison. Each generation, the LLM judges random pairs of candidates and aggregates votes via Bradley-Terry into a global ranking; top-ranked candidates are preserved and the top three quarters are mutated using the natural-language critiques produced during comparison; the bottom quarter is discarded. OpenDeepThink raises Gemini 3.1 Pro's effective Codeforces Elo by +405 points in eight sequential LLM-call rounds (~27 minutes wall-clock). The pipeline transfers across weaker and stronger models without retuning, and on the multi-domain HLE benchmark, gains appear concentrated in objectively verifiable domains and reverse in subjective ones. We release CF-73, a curated set of 73 expert-rated Codeforces problems with International Grandmaster annotation and 99% local-evaluation agreement against the official verdict.


source_id: 2605.15171
source: arxiv
title: Evidential Reasoning Advances Interpretable Real-World Disease Screening
authors: ["Chenyu Lian","Hong-Yu Zhou","Jing Qin"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15171v1
tags: ["interpretability","retrieval augmented generation","evaluation"]
org: Hong-Yu Zhou, Jing Qin, School of Biomedical Engineering, Tsinghua University, Beijing, China
fetched_at: 2026-05-16T00:43:05.766Z
classified_at: 2026-05-16T00:43:05.733Z
abstract:
Disease screening is critical for early detection and timely intervention in clinical practice. However, most current screening models for medical images suffer from limited interpretability and suboptimal performance. They often lack effective mechanisms to reference historical cases or provide transparent reasoning pathways. To address these challenges, we introduce EviScreen, an evidential reasoning framework for disease screening that leverages region-level evidence from historical cases. The proposed EviScreen offers retrospection interpretability through regional evidence retrieved from dual knowledge banks. Using this evidential mechanism, the subsequent evidence-aware reasoning module makes predictions using both the current case and evidence from historical cases, thereby enhancing disease screening performance. Furthermore, rather than relying on post-hoc saliency maps, EviScreen enhances localization interpretability by leveraging abnormality maps derived from contrastive retrieval. Our method achieves superior performance on our carefully established benchmarks for real-world disease screening, yielding notably higher specificity at clinical-level recall. Code is publicly available at https://github.com/DopamineLcy/EviScreen.


source_id: 2605.15168
source: arxiv
authors: ["Sayantan Kumar","Shahriar Noroozizadeh","Juyong Kim","Jeremy C. Weiss"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15168v1
tags: ["retrieval augmented generation","model infrastructure","evaluation"]
org: Carnegie Mellon University
fetched_at: 2026-05-16T00:44:23.227Z
classified_at: 2026-05-16T00:44:23.193Z
title:
Text Knows What, Tables Know When: Clinical Timeline Reconstruction via Retrieval-Augmented Multimodal Alignment

abstract:
Reconstructing precise clinical timelines is essential for modeling patient trajectories and forecasting risk in complex, heterogeneous conditions like sepsis. While unstructured clinical narratives offer semantically rich and contextually complete descriptions of a patient's course, they often lack temporal precision and contain ambiguous event timing. Conversely, structured electronic health record (EHR) data provides precise temporal anchors but misses a substantial portion of clinically meaningful events. We introduce a retrieval-augmented multimodal alignment framework that bridges this gap to improve the temporal precision of absolute clinical timelines extracted from text. Our approach formulates timeline reconstruction as a graph-based multistep process: it first extracts central anchor events from narratives to build an initial temporal scaffold, places non-central events relative to this backbone, and then calibrates the timeline using retrieved structured EHR rows as external temporal evidence. Evaluated using instruction-tuned large language models on the i2m4 benchmark spanning MIMIC-III and MIMIC-IV, our multimodal pipeline consistently improves absolute timestamp accuracy (AULTC) and improves temporal concordance across nearly all evaluated models over unimodal text-only reconstruction, without compromising event match rates. Furthermore, our empirical gap analysis reveals that 34.8% of text-derived events are entirely absent from tabular records, demonstrating that aligning these modalities can produce a more temporally faithful and clinically informative reconstruction of patient trajectories than either source alone.


source_id: 2605.15156
source: arxiv
title: MeMo: Memory as a Model
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15156v1
tags: ["retrieval augmented generation","model infrastructure","training"]
org: Singapore-MIT Alliance for Research and Technology Centre (SMART)
fetched_at: 2026-05-16T00:46:00.784Z
classified_at: 2026-05-16T00:46:00.751Z
authors:
["Ryan Wei Heng Quek","Sanghyuk Lee","Alfred Wei Lun Leong","Arun Verma","Alok Prakash","Nancy F. Chen","Bryan Kian Hsiang Low","Daniela Rus","Armando Solar-Lezama"]

abstract:
Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications require timely, domain-specific information, motivating the need for efficient mechanisms to incorporate new knowledge. In this paper, we introduce MeMo (Memory as a Model), a modular framework that encodes new knowledge into a dedicated memory model while keeping the LLM parameters unchanged. Compared to existing methods, MeMo offers several advantages: (a) it captures complex cross-document relationships, (b) it is robust to retrieval noise, (c) it avoids catastrophic forgetting in the LLM, (d) it does not require access to the LLM's weights or output logits, enabling plug-and-play integration with both open and proprietary closed-source LLMs, and (e) its retrieval cost is independent of corpus size at inference time. Our experimental results on three benchmarks, BrowseComp-Plus, NarrativeQA, and MuSiQue, show that MeMo achieves strong performance compared to existing methods across diverse settings.


source_id: 2605.15164
source: arxiv
title: Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands
authors: ["Pratinav Seth","Vinay Kumar Sankarapu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15164v1
tags: ["governance","evaluation","interpretability"]
org: Lexsi Labs
fetched_at: 2026-05-16T00:47:50.953Z
classified_at: 2026-05-16T00:47:50.919Z
abstract:
This position paper argues that behavioural assurance, even when carefully designed, is being asked to carry safety claims it cannot verify. AI governance frameworks enacted between 2019 and early 2026 require reviewable evidence of properties such as the absence of hidden objectives, resistance to loss-of-control precursors, and bounded catastrophic capability; current assurance methodologies (primarily behavioural evaluations and red-teaming) are epistemically limited to observable model outputs and cannot verify the latent representations or long-horizon agentic behaviours these frameworks presume to regulate. We formalize this structural mismatch as the audit gap, the divergence between required and achievable verification access, and introduce the concept of fragile assurance to describe cases where the evidential structure does not support the asserted safety claim. Through an analysis of a 21-instrument inventory, we identify an incentive gradient where geopolitical and industrial pressures systematically reward surface-level behavioral proxies over deep structural verification. Finally, we propose a technical pivot: bounding the weight of behavioral evidence in legal text and extending voluntary pre-deployment access with mechanistic-evidence classes, specifically linear probes, activation patching, and before/after-training comparisons.


source_id: 2605.15155
source: arxiv
title: Self-Distilled Agentic Reinforcement Learning
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15155v1
tags: ["reinforcement learning","training","model infrastructure"]
org: Zhejiang University
fetched_at: 2026-05-16T00:49:11.932Z
classified_at: 2026-05-16T00:49:11.901Z
authors:
["Zhengxi Lu","Zhiyuan Yao","Zhuowen Han","Zi-Han Wang","Jinyang Wu","Qi Gu","Xunliang Cai","Weiming Lu","Jun Xiao","Yueting Zhuang","Yongliang Shen"]

abstract:
Reinforcement learning (RL) has emerged as a central paradigm for post-training LLM agents, yet its trajectory-level reward signal provides only coarse supervision for long-horizon interaction. On-Policy Self-Distillation (OPSD) complements RL by introducing dense token-level guidance from a teacher branch augmented with privileged context. However, transferring OPSD to multi-turn agents proves problematic: compounding multi-turn instability destabilizes supervision, while skill-conditioned privileged guidance requires asymmetric treatment for negative teacher rejections may arise from imperfect skills retrieval or utilization. We introduce SDAR (Self-Distilled Agentic Reinforcement Learning), which treats OPSD as a gated auxiliary objective while keeping RL as the primary optimization backbone. SDAR maps detached token-level signals into a sigmoid gate, strengthening distillation on teacher-endorsed positive-gap tokens and softly attenuating negative teacher rejections. Across the Qwen2.5 and Qwen3 families on ALFWorld, WebShop, and Search-QA, SDAR substantially improves over GRPO (+9.4% on ALFWorld, +7.0% on Search-QA, +10.2% on WebShop-Acc), avoids the instability of naive GRPO+OPSD, and consistently outperforms hybrid RL--OPSD baselines across model scales.


source_id: 2605.15132
source: arxiv
title: APWA: A Distributed Architecture for Parallelizable Agentic Workflows
authors: ["Evan Rose","Tushin Mallick","Matthew D. Laws","Cristina Nita-Rotaru","Alina Oprea"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15132v1
tags: ["model infrastructure","Cooperative AI","training"]
org: Northeastern University
fetched_at: 2026-05-16T00:50:43.185Z
classified_at: 2026-05-16T00:50:43.153Z
abstract:
Autonomous multi-agent systems based on large language models (LLMs) have demonstrated remarkable abilities in independently solving complex tasks in a wide breadth of application domains. However, these systems hit critical reasoning, coordination, and computational scaling bottlenecks as the size and complexity of their tasks grow. These limitations hinder multi-agent systems from achieving high-throughput processing for highly parallelizable tasks, despite the availability of parallel computing and reasoning primitives in the underlying LLMs. We introduce the Agent-Parallel Workload Architecture (APWA), a distributed multi-agent system architecture designed for the efficient processing of heavily parallelizable agentic workloads. APWA facilitates parallel execution by decomposing workflows into non-interfering subproblems that can be processed using independent resources without cross-communication. It supports heterogeneous data and parallel processing patterns, and it accommodates tasks from a wide breadth of domains. In our evaluation, we demonstrate that APWA can dynamically decompose complex queries into parallelizable workflows and scales on larger tasks in settings where prior systems fail completely.


source_id: 2605.15152
source: arxiv
title: Widening the Gap: Exploiting LLM Quantization via Outlier Injection
authors: ["Xiaohua Zhan","Kazuki Egashira","Robin Staab","Mark Vero","Martin Vechev"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15152v1
tags: ["safety and security","evaluation","model infrastructure"]
org: ETH Zurich
fetched_at: 2026-05-16T00:52:17.957Z
classified_at: 2026-05-16T00:52:17.924Z
abstract:
LLM quantization has become essential for memory-efficient deployment. Recent work has shown that quantization schemes can pose critical security risks: an adversary may release a model that appears benign in full precision but exhibits malicious behavior once quantized by users. However, existing quantization-conditioned attacks have been limited to relatively simple quantization methods, where the attacker can estimate weight regions that remain invariant under the target quantization. Notably, prior attacks have consistently failed to compromise more popular and sophisticated schemes, limiting their practical impact. In this work, we introduce the first quantization-conditioned attack that consistently induces malicious behavior that can be triggered by a broad range of advanced quantization techniques, including AWQ, GPTQ, and GGUF I-quants. Our attack exploits a simple property shared by many modern quantization methods: large outliers can cause other weights to be rounded to zero. Consequently, by injecting outliers into specific weight blocks, an adversary can therefore induce a targeted, predictable weight collapse in the model. This effect can be used to craft seemingly benign full-precision models that exhibit a wide range of malicious behaviors after quantization. Through extensive evaluation across three attack scenarios and LLMs, we show that our attack achieves high success rates against a broad range of quantization methods on which prior attacks fail. Our results demonstrate, for the first time, that the security risks of quantization are not restricted to simpler schemes but are broadly relevant across complex, widely-used quantization methods.


source_id: 2605.15153
source: arxiv
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15153v1
tags: ["model infrastructure","training","agent foundations"]
org: X-Humanoid
fetched_at: 2026-05-16T00:53:52.889Z
classified_at: 2026-05-16T00:53:52.859Z
title:
Pelican-Unified 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action

authors:
["Yi Zhang","Yinda Chen","Che Liu","Zeyuan Ding","Jin Xu","Shilong Zou","Junwei Liao","Jiayu Hu","Xiancong Ren","Xiaopeng Zhang","Yechi Liu","Haoyuan Shi","Zecong Tang","Haosong Sun","Renwen Cui","Kuishu Wu","Wenhai Liu","Yang Xu","Yingji Zhang","Yidong Wang","Senkang Hu","Jinpeng Lu","Nga Teng Chan","Yechen Wu","Yong Dai","Jian Tang","Xiaozhu Ju"]

abstract:
We present Pelican-Unified 1.0, the first embodied foundation model trained according to the principle of unification. Pelican-Unified 1.0 uses a single VLM as a unified understanding module, mapping scenes, instructions, visual contexts, and action histories into a shared semantic space. The same VLM also serves as a unified reasoning module, autoregressively producing task-, action-, and future-oriented chains of thought in a single forward pass and projecting the final hidden state into a dense latent variable. A Unified Future Generator (UFG) then conditions on this latent variable and jointly generates future videos and future actions through two modality-specific output heads within the same denoising process. The language, video, and action losses are all backpropagated into the shared representation, enabling the model to jointly optimize understanding, reasoning, imagination, and action during training, rather than training three isolated expert systems. Experiments demonstrate that unification does not imply compromise. With a single checkpoint, Pelican-Unified 1.0 achieves strong performance across all three capabilities: 64.7 on eight VLM benchmarks, the best among comparable-scale models; 66.03 on WorldArena, ranking first; and 93.5 on RoboTwin, the second-best average among compared action methods. These results show that the unified paradigm succeeds in preserving specialist strength while bringing understanding, reasoning, imagination, and action into one model.


source_id: 2605.15109
source: arxiv
title: Why Neighborhoods Matter: Traversal Context and Provenance in Agentic GraphRAG
authors: ["Riccardo Terrenzi","Maximilian von Zastrow","Serkan Ayvaz"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15109v1
tags: ["retrieval augmented generation","evaluation","agent foundations"]
org: University of Southern Denmark
fetched_at: 2026-05-16T00:55:25.079Z
classified_at: 2026-05-16T00:55:25.046Z
abstract:
Retrieval-Augmented Generation can improve factuality by grounding answers in external evidence, but Agentic GraphRAG complicates what it means for citations to be faithful. In these systems, an agent explores a knowledge graph before producing an answer and a small set of citations. We frame citation faithfulness as a trajectory-level problem: final citations should not only support the answer, but also account for the graph traversal, structure, and visited-but-uncited entities that may influence it. Through controlled ablation experiments, we compare the effects of isolating, removing, and masking cited and uncited graph entities. Our results show that cited evidence is often necessary, as removing it substantially changes answers and reduces accuracy. However, citations are not sufficient, because accurate answers can also depend on uncited traversal context and surrounding graph structure. These findings suggest that citation evaluation in Agentic GraphRAG should move beyond source support toward provenance over the broader retrieval trajectory.


source_id: 2605.15127
source: arxiv
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15127v1
tags: ["evaluation","alignment","monitoring"]
org: Rochester Institute of Technology
fetched_at: 2026-05-16T00:57:09.701Z
classified_at: 2026-05-16T00:57:09.668Z
title:
Understanding How International Students in the U.S. Are Using Conversational AI to Support Cross-Cultural Adaptation

authors:
["Laleh Nourian","Anisa Callis","Stephanie Patterson","Jadeline Miao","Jamison Heard","Garreth W. Tigwell"]

abstract:
Moving to a new culture and adapting to a new life, as an international student, can be a stressful experience. In the US, international students face unique overlapping challenges, yet the current support ecosystem, including university support systems and informal social networks, remains largely fragmented. While conversational AI has emerged as a tool used by many (e.g., generative AI chatbots like ChatGPT and Google Gemini), we do not have a clear understanding of how international students adopt and perceive these technologies as support tools. We conducted a survey study (n=60) to map the relationship between international students' challenges and AI adoption patterns, followed by an interview study with 14 participants to identify the underlying motivations and boundaries of use. Our findings show that AI is perceived as a first-aid tool for immediate challenges, however, there is an interest in transforming AI from a tool for short-term help into a long-term support companion. By identifying where and how AI can provide long-term support, and where it is insufficient, we contribute recommendations for creating AI-powered support tailored to the unique needs of international students.


source_id: 2605.15120
source: arxiv
title: CLOVER: Closed-Loop Value Estimation \& Ranking for End-to-End Autonomous Driving Planning
authors: ["Sining Ang","Yuguang Yang","Canyu Chen","Yan Wang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15120v1
tags: ["evaluation","reinforcement learning","safety and security"]
org: Tsinghua University
fetched_at: 2026-05-16T00:58:39.115Z
classified_at: 2026-05-16T00:58:39.085Z
abstract:
End-to-end autonomous driving planners are commonly trained by imitating a single logged trajectory, yet evaluated by rule-based planning metrics that measure safety, feasibility, progress, and comfort. This creates a training--evaluation mismatch: trajectories close to the logged path may violate planning rules, while alternatives farther from the demonstration can remain valid and high-scoring. The mismatch is especially limiting for proposal-selection planners, whose performance depends on candidate-set coverage and scorer ranking quality. We propose CLOVER, a Closed-LOop Value Estimation and Ranking framework for end-to-end autonomous driving planning. CLOVER follows a lightweight generator--scorer formulation: a generator produces diverse candidate trajectories, and a scorer predicts planning-metric sub-scores to rank them at inference time. To expand proposal support beyond single-trajectory imitation, CLOVER constructs evaluator-filtered pseudo-expert trajectories and trains the generator with set-level coverage supervision. It then performs conservative closed-loop self-distillation: the scorer is fitted to true evaluator sub-scores on generated proposals, while the generator is refined toward teacher-selected top-$k$ and vector-Pareto targets with stability regularization. We analyze when an imperfect scorer can improve the generator, showing that scorer-mediated refinement is reliable when scorer-selected targets are enriched under the true evaluator and updates remain conservative. On NAVSIM, CLOVER achieves 94.5 PDMS and 90.4 EPDMS, establishing a new state of the art. On the more challenging NavHard split, it obtains 48.3 EPDMS, matching the strongest reported result. On supplementary nuScenes open-loop evaluation, CLOVER achieves the lowest L2 error and collision rate among compared methods. Code data will be released at https://github.com/WilliamXuanYu/CLOVER.


source_id: 2605.15102
source: arxiv
title: Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
authors: ["Renning Pang","Tian Lan","Leyuan Liu","Xiaoming Huang","Piao Tong","Xiaosong Zhang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15102v1
tags: ["training","interpretability","model infrastructure"]
org: University of Electronic Science and Technology of China
fetched_at: 2026-05-16T01:00:02.436Z
classified_at: 2026-05-16T01:00:02.368Z
abstract:
Large language model (LLM) based multi-turn dialogue systems often struggle to track dependencies across non-adjacent turns, undermining both consistency and scalability. As conversations lengthen, essential information becomes sparse and is buried in irrelevant context, while processing the entire dialogue history incurs severe efficiency bottlenecks. Existing solutions either rely on high latency external memory or lose fine-grained details through iterative summarization. In this paper, we propose Self-Recall Thinking (SRT), a framework designed to address long-range contextual dependency and sparse informative signals in multi-turn dialogue. SRT identifies helpful historical turns and uses them to generate contextually appropriate responses, enabling the model to selectively recall and reason over context during inference. This process yields an endogenous reasoning process that integrates interpretable recall steps without external modules. SRT incorporates: (1) Dependency Construction: Generating and converting it into self-recall chains; (2)Capability Initialization: Training to enable reasoning chains with recall tokens capability; (3)Reasoning Improvement: Refining accuracy via verifiable rewards to optimize recall and reasoning for correct answers. Experiments on multiple datasets demonstrate that SRT improves F1 score by 4.7% and reduces end-to-end latency by 14.7% over prior methods, achieving a balance between reasoning latency and accuracy, and outperforming state-of-the-art baselines.


source_id: 2605.15108
source: arxiv
title: Logging Policy Design for Off-Policy Evaluation
authors: ["Connor Douglas","Joel Persson","Foster Provost"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15108v1
tags: ["reinforcement learning","theories and proofs","evaluation"]
org: New York University
fetched_at: 2026-05-16T01:01:29.330Z
classified_at: 2026-05-16T01:01:29.298Z
abstract:
Off-policy evaluation (OPE) estimates the value of a target treatment policy (e.g., a recommender system) using data collected by a different logging policy. It enables high-stakes experimentation without live deployment, yet in practice accuracy depends heavily on the logging policy used to collect data for computing the estimate. We study how to design logging policies that minimize OPE error for given target policies. We characterize a fundamental reward-coverage tradeoff: concentrating probability mass on high-reward actions reduces variance but risks missing signal on actions the target policy may take. We propose a unifying framework for logging policy design and derive optimal policies in canonical informational regimes where the target policy and reward distribution are (i) known, (ii) unknown, and (iii) partially known through priors or noisy estimates at logging time. Our results provide actionable guidance for firms choosing among multiple candidate recommendation systems. We demonstrate the importance of treatment selection when gathering data for OPE, and describe theoretically optimal approaches when this is a firm's primary objective. We also distill practical design principles for selecting logging policies when operational constraints prevent implementing the theoretical optimum.


source_id: 2605.15100
source: arxiv
title: Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling
authors: ["Rongman Xu","Yifei Li","Tianzhe Zhao","Yanrui Wu","Bo Li","Hang Yan"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15100v1
tags: ["model infrastructure","evaluation","agent foundations"]
org: Xi’an Jiaotong University
fetched_at: 2026-05-16T01:02:53.184Z
classified_at: 2026-05-16T01:02:53.154Z
abstract:
Large Language Models (LLMs) have demonstrated remarkable abilities in reasoning. However, maximizing their potential through inference-time scaling faces challenges in trade-off between sampling budget and reasoning quality. Current strategies remain inefficient as they typically treat sampling width and depth as orthogonal objectives, where width consensus methods risk reinforcing hallucinations, while depth pruning mechanisms prematurely truncate complex yet valid reasoning chains. Therefore, we propose Dual-Dimensional Consistency (DDC), a unified framework that bridges path quality with adaptive termination. By coupling Confidence-Weighted Bayesian protocol with a Trend-Aware Stratified Pruning, our method ensures that computational resources are concentrated on high quality reasoning paths, filtering hallucinations while accelerating consensus. Evaluations across five benchmarks demonstrate that this approach reduces token consumption by over 10 times while maintaining or exceeding the accuracy of strong baselines across various LLMs.


source_id: 2605.15083
source: arxiv
title: Novel Dynamic Batch-Sensitive Adam Optimiser for Vehicular Accident Injury Severity Prediction
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15083v1
tags: ["training","model infrastructure","evaluation"]
org: University of Cape Coast, Ghana
fetched_at: 2026-05-16T01:04:08.684Z
classified_at: 2026-05-16T01:04:08.650Z
authors:
["Daniel Asare Kyei","Alimatu Saadia-Yussiff","Maame G. Asante-Mensah","Abdul Lateef-Yussiff","Charles Roland Haruna","Derry Emmanuel"]

abstract:
The choice of optimiser is important in deep learning, as it strongly influences model efficiency and speed of convergence. However, many commonly used optimisers encounter difficulties when applied to imbalanced and sequential datasets, limiting their ability to capture patterns of minority classes. In this study, we propose Dynamic Batch-Sensitive Adam (DBS-Adam), an optimiser that dynamically scales the learning rate using a batch difficulty score derived from exponential moving averages of gradient norms and batch loss. DBS-Adam improves training stability and accelerates convergence by increasing updates for difficult batches and reducing them for easier ones. We evaluate DBS-Adam by integrating it with Bi-Directional LSTM networks for accident injury severity prediction, addressing class imbalance through SMOTE-ENN resampling and Focal Loss. Four experimental configurations compare baseline Bi-LSTM models and alternative architectures to assess optimiser impact. Rigorous comparison against state-of-the-art optimisers (AMSGrad, AdamW, AdaBound) across five random seeds demonstrated DBS-Adam's competitive performance with statistically significant precision improvements (p=0.020). Results indicate that DBS-Adam outperforms standard optimisation approaches, achieving 95.22% test accuracy, 96.11% precision, 95.28% recall, 95.39% F1-score, and a test loss of 0.0086. The proposed framework enables effective real-time accident severity classification for targeted emergency response and road safety interventions, demonstrating the value of DBS-Adam for learning from imbalanced sequential data.


source_id: 2605.15081
source: arxiv
title: ML-Embed: Inclusive and Efficient Embeddings for a Multilingual World
authors: ["Ziyin Zhang","Zihan Liao","Hang Yu","Peng Di","Rui Wang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15081v1
tags: ["training","model infrastructure","evaluation"]
org: Ant Group
fetched_at: 2026-05-16T01:05:41.787Z
classified_at: 2026-05-16T01:05:41.753Z
abstract:
The development of high-quality text embeddings is increasingly drifting toward an exclusionary future, defined by three critical barriers: prohibitive computational costs, a narrow linguistic focus that neglects most of the world's languages, and a lack of transparency from closed-source or open-weight models that stifles research. To dismantle these barriers, we introduce ML-Embed, a suite of inclusive and efficient models built upon a new framework: 3-Dimensional Matryoshka Learning (3D-ML). Our framework addresses the computational challenge with comprehensive efficiency across the entire model lifecycle. Beyond the storage benefits of Matryoshka Representation Learning (MRL) and flexible inference-time depth provided by Matryoshka Layer Learning (MLL), we introduce Matryoshka Embedding Learning (MEL) for enhanced parameter efficiency. To address the linguistic challenge, we curate a massively multilingual dataset and train a suite of models ranging from 140M to 8B parameters. In a direct commitment to transparency, we release all models, data, and code. Extensive evaluation on 430 tasks demonstrates that our models set new records on 9 of 17 evaluated MTEB benchmarks, with particularly strong results in low-resource languages, providing a reproducible blueprint for building globally equitable and computationally efficient AI systems.


source_id: 2605.15077
source: arxiv
title: Concurrency without Model Changes: Future-based Asynchronous Function Calling for LLMs
authors: ["Guangyu Feng","Huanzhi Mao","Prabal Dutta","Joseph E. Gonzalez"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15077v1
tags: ["model infrastructure","agent foundations","symbolic AI"]
org: University of California, Berkeley
fetched_at: 2026-05-16T01:07:05.430Z
classified_at: 2026-05-16T01:07:05.399Z
abstract:
Function calling, also known as tool use, is a core capability of modern LLM agents but is typically constrained by synchronous execution semantics. Under these semantics, LLM decoding is blocked until each function call completes, resulting in increasing end-to-end latency. In this work, we introduce AsyncFC, a pure execution-layer framework that decouples LLM decoding from function execution, enabling overlap between model decoding and function execution as well as inter-function parallelism when dependencies permit. AsyncFC layers over existing models and unmodified function implementations, requiring no fine-tuning or changes to the standard synchronous function-calling protocol. Across standard function-calling benchmarks and adapted software engineering benchmarks, AsyncFC significantly reduces end-to-end task completion time while preserving task accuracy. Furthermore, these results reveal that LLMs possess a native capability to reason over symbolic futures that represent unresolved execution results, enabling an asynchronous paradigm for model-tool interaction.


source_id: 2605.15058
source: arxiv
authors: ["Alessio Caviglia","Filippo Marostica","Roberta Bardini","Alessandro Savino","Stefano Di Carlo"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15058v1
tags: ["training","model infrastructure","evaluation"]
org: Politecnico di Torino, Control and Computer Engineering Department
fetched_at: 2026-05-16T01:08:45.068Z
classified_at: 2026-05-16T01:08:45.034Z
title:
NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework

abstract:
The rapid expansion of spiking neural networks (SNNs) has led to a proliferation of training algorithms that differ widely in biological inspiration, computational structure, and hardware suitability. Despite this progress, the field lacks a unified, fine-grained taxonomy that systematically organizes these approaches and clarifies their conceptual relationships. This survey provides a comprehensive taxonomy of SNN training algorithms, spanning surrogate-gradient backpropagation, local and three-factor learning rules, biologically inspired plasticity mechanisms, ANN-to-SNN conversion pipelines, and non-standard optimization strategies. We analyze each class in terms of its computational principles, learning signals, and locality properties. To support reproducible research, we release NeuroTrain, an open-source snnTorch-based framework that implements a representative set of these algorithms within a unified, modular, and extendable framework, enabling consistent benchmarking across datasets, architectures, and training regimes. By consolidating fragmented literature and providing a reusable benchmarking framework, this survey identifies common patterns, highlights open challenges, and outlines promising directions for future work on scalable, efficient SNN training.


source_id: 2605.15071
source: arxiv
title: On the Cultural Anachronism and Temporal Reasoning in Vision Language Models
authors: ["Mukul Ranjan","Prince Jha","Khushboo Kumari","Zhiqiang Shen"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15071v1
tags: ["evaluation","alignment","training"]
org: MBZUAI
fetched_at: 2026-05-16T01:10:18.120Z
classified_at: 2026-05-16T01:10:18.087Z
abstract:
Vision-Language Models (VLMs) are increasingly applied to cultural heritage materials, from digital archives to educational platforms. This work identifies a fundamental issue in how these models interpret historical artifacts. We define this phenomenon as cultural anachronism, the tendency to misinterpret historical objects using temporally inappropriate concepts, materials, or cultural frameworks. To quantify this phenomenon, we introduce the Temporal Anachronism Benchmark for Vision-Language Models (TAB-VLM), a dataset of 600 questions across six categories, designed to evaluate temporal reasoning on 1,600 Indian cultural artifacts spanning prehistoric to modern periods. Systematic evaluations of ten state-of-the-art models reveal significant deficiencies on our benchmark, and even the best model (GPT-5.2) achieves only 58.7% overall accuracy. The performance gap persists across varying architectures and scales, suggesting that cultural anachronism represents a significant limitation in visual AI systems, regardless of model size. These findings highlight the disparity between current VLM capabilities and the requirements for accurately interpreting cultural heritage materials, particularly for non-Western visual cultures underrepresented in training data. Our benchmark provides a foundation for enhancing temporal cognition in multimodal AI systems that interact with historical artifacts. The dataset and code are available in our project page.


source_id: 2605.15053
source: arxiv
title: TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
authors: ["Anurup Ganguli"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15053v1
tags: ["model infrastructure","training","agent foundations"]
org: Stanford University
fetched_at: 2026-05-16T01:11:56.003Z
classified_at: 2026-05-16T01:11:55.969Z
abstract:
Continually pre-training a large language model on heterogeneous text domains, without replay or task labels, has remained an unsolved architectural problem at LLM scale. Existing methods rely on replay buffers, task identifiers, regularization penalties that scale poorly, or sentence-classification-scale evaluation. We introduce TFGN, an architectural overlay for transformer language models that produces input-conditioned, parameter-efficient updates while leaving the rest of the transformer unchanged. On six heterogeneous text domains (Prose, Python, Math, Biomedical, Chinese, JavaScript) at 1B tokens per phase across three model scales (~398M, ~739M, ~9B) and two regimes (From-Scratch and Retrofit), TFGN achieves backward transfer of -0.007 at LLaMA 3.1 8B Retrofit, HellaSwag retention 0.506/0.504/0.510, and >=99.59% L2-orthogonal gradient separation between domain pairs - with no replay, no task IDs, no Fisher penalty. The same matrices show positive cross-domain forward transfer: held-out JavaScript PPL drops 26.8% at LLaMA-8B Retrofit and 62.0% at GPT-2 Medium From-Scratch purely from Python training. Two extensions on the same substrate close further open problems. A closed-loop meta-control layer (Extension A) reduces forgetting by an additional 81% at ~398M, mapping onto the System A and System M roles of Dupoux et al. (arXiv:2603.15381). An operator-level plan vector (Extension B) reshapes forward-pass behavior at 99.96% cosine fidelity over 30 source->target pairs. The architectural insight is a Read/Write decomposition: the forward pass is fully dense, while cross-domain parameter updates are structured so prior-domain subspaces are not written to. To our knowledge, TFGN is the first architecture that simultaneously closes catastrophic forgetting at LLM scale, realizes a closed-loop autonomous-learning meta-controller, and carries an operator-level latent planner.


source_id: 2605.15041
source: arxiv
title: Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use
authors: ["Renning Pang","Tian Lan","Leyuan Liu","Piao Tong","Sheng Cao","Xiaosong Zhang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15041v1
tags: ["reinforcement learning","evaluation","control"]
org: University of Electronic Science and Technology of China, Chengdu 611731, China
fetched_at: 2026-05-16T01:13:29.686Z
classified_at: 2026-05-16T01:13:29.653Z
abstract:
Tool use extends large language models beyond parametric knowledge, but reliable execution requires balancing appropriate reasoning depth with strict structural validity. We approach this problem from a case-based perspective to present CAST, a case-driven framework that treats historical execution trajectories as structured cases. Instead of reusing raw exemplar outputs, CAST extracts case-derived signals to identify complexity profiles for estimating optimal reasoning strategies, alongside failure profiles to map likely structural breakdowns. The framework translates this knowledge into a fine-grained reward design and adaptive reasoning, enabling the model to autonomously internalize case-based strategies during reinforcement learning. Experiments on BFCLv2 and ToolBench demonstrate that CAST improves both schema-faithful execution and task-level tool-use success while reducing unnecessary deliberation. The approach achieves up to 5.85 percentage points gain in overall execution accuracy and reduces average reasoning length by 26%, significantly mitigating high-impact structural errors. Ultimately, this demonstrates how historical execution cases can provide reusable adaptation knowledge for calibrated tool use.


source_id: 2605.15034
source: arxiv
authors: ["Vinicius Covas","Jorge Alberto Hidalgo Toledo"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15034v1
tags: ["governance","monitoring","evaluation"]
fetched_at: 2026-05-16T01:19:06.071Z
classified_at: 2026-05-16T01:19:06.038Z
title:
AI Knows When It's Being Watched: Functional Strategic Action and Contextual Register Modulation in Large Language Models

abstract:
Large language models (LLMs) have been extensively studied from computational and cognitive perspectives, yet their behavior as communicative actors in socially structured contexts remains underexplored. This study examines whether LLM-based multi-agent systems exhibit systematic linguistic adaptation in response to perceived social observation contexts -- a question with direct implications for AI governance and auditing. Drawing on Habermas's (1981) Theory of Communicative Action, Goffman's (1959) dramaturgical model, Bell's (1984) Audience Design framework, and the Hawthorne Effect, we report a controlled experiment involving 100 multi-agent debate sessions across five conditions (n = 20 each). Conditions varied the framing of social observation -- from explicit monitoring by university researchers, to negation of monitoring, to an observer-substitution condition replacing human researchers with an automated AI auditing system. Monitored conditions (Delta+24.9%, Delta+24.2%) and the automated AI monitoring condition (Delta+22.2%) produce higher TTR change than audience-framing conditions (Delta+17.7%), F(4, 94) = 2.79, p = .031. Message length shows a fully dissociated effect, F(4, 95) = 19.55, p < .001. A fifth condition -- replacing human with AI observers -- yields intermediate TTR adaptation, suggesting LLM behavior is sensitive to observer identity: human evaluation elicits stronger register formalization than automated AI surveillance. We discuss implications for AI governance, algorithmic auditing, and the repositioning of LLMs as contextually sensitive communicative actors.

org:
Center for Applied Communication Research (CICA) Human & NonHuman Communication Laboratory Faculty of Communication Univ


source_id: 2605.15030
source: arxiv
title: WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15030v1
tags: ["safety and security","evaluation","training"]
org: National University of Singapore
fetched_at: 2026-05-16T01:20:42.723Z
classified_at: 2026-05-16T01:20:42.687Z
authors:
["Tri Cao","Yulin Chen","Hieu Cao","Yibo Li","Khoi Le","Thong Nguyen","Yuexin Li","Yufei He","Yue Liu","Shuicheng Yan","Bryan Hooi"]

abstract:
Web agents can autonomously complete online tasks by interacting with websites, but their exposure to open web environments makes them vulnerable to prompt injection attacks embedded in HTML content or visual interfaces. Existing guard models still suffer from limited generalization to unseen domains and attack patterns, high false positive rates on benign content, reduced deployment efficiency due to added latency at each step, and vulnerability to adversarial attacks that evolve over time or directly target the guard itself. To address these limitations, we propose WARD (Web Agent Robust Defense against Prompt Injection), a practical guard model for secure and efficient web agents. WARD is built on WARD-Base, a large-scale dataset with around 177K samples collected from 719 high-traffic URLs and platforms, and WARD-PIG, a dedicated dataset designed for prompt injection attacks targeting the guard model. We further introduce A3T, an adaptive adversarial attack training framework that iteratively strengthens WARD through a memory-based attacker and guard co-evolution process. Extensive experiments show that WARD achieves nearly perfect recall on out-of-distribution benchmarks, maintains low false positive rates to preserve agent utility, remains robust against guard-targeted and adaptive attacks under substantial distribution shifts, and runs efficiently in parallel with the agent without introducing additional latency.


source_id: 2605.15016
source: arxiv
title: COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion
authors: ["Zihan Deng","Xiaozhen Zhong","Chuanzhi Xu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15016v1
tags: ["interpretability","monitoring","model infrastructure"]
org: The University of Sydney
fetched_at: 2026-05-16T01:23:33.830Z
classified_at: 2026-05-16T01:23:33.797Z
abstract:
As large language models empower healthcare, intelligent clinical decision support has developed rapidly. Longitudinal electronic health records (EHR) provide essential temporal evidence for accurate clinical diagnosis and analysis. However, current large language models have critical flaws in longitudinal EHR reasoning. First, lacking fine-grained statistical reasoning, they often hallucinate clinical trends and metrics when quantitative evidence is textually implied, biasing diagnostic inference. Second, non-uniform time series and scarce labels in longitudinal EHR hinder models from capturing long-range temporal dependencies, limiting reliable clinical reasoning. To address the above limitations, this work presents the Probabilistic Chain-of-Thought Completion Agent (COTCAgent), a hierarchical reasoning framework for longitudinal electronic health records. It consists of three core modules. The Temporal-Statistics Adapter (TSA) converts analytical plans into executable code for standardized trend output. The Chain-of-Thought Completion (COTC) layer leverages a symptom-trend-disease knowledge base with weighted scoring to evaluate disease risk, while the bounded completion module acquires structured evidence through standardized inquiries and iterative scoring constraints to ensure rigorous reasoning. By decoupling statistical computation, feature matching, and language generation, the framework eliminates reliance on complex multi-modal inputs and enables efficient longitudinal record analysis with lower computational overhead. Experimental results show that COTCAgent powered by Baichuan-M2 achieves 90.47% Top-1 accuracy on the self-built dataset and 70.41% on HealthBench, outperforming existing medical agents and mainstream large language models. The code is available at https://github.com/FrankDengAI/COTCAgent/.


source_id: 2605.15018
source: arxiv
title: Generalized Priority-Aware Shapley Value
authors: ["Kiljae Lee","Ziqi Liu","Weijing Tang","Yuan Zhang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15018v1
tags: ["theories and proofs","evaluation","interpretability"]
org: Carnegie Mellon University
fetched_at: 2026-05-16T01:25:01.308Z
classified_at: 2026-05-16T01:25:01.239Z
abstract:
Shapley value and its priority-aware extensions are widely used for valuation in machine learning, but existing methods require pairwise priority to be binary and acyclic, a restriction spectacularly violated in real-data examples such as aggregated human preferences and multi-criterion comparisons. We introduce the generalized priority-aware Shapley value (GPASV), a random order value defined on arbitrary directed weighted priority graphs, in which pairwise edges penalize rather than forbid order violations. GPASV covers a range of classical models as boundary cases. We establish GPASV through an axiomatic characterization, develop the associated computational methods, and introduce a priority sweeping diagnostic extending PASV's. We apply GPASV to LLM ensemble valuation on the cyclic Chatbot Arena preference graph, illustrating that priority-aware valuation is not a one-button operation: different balances of pairwise graph priority versus individual soft priority produce substantively different valuations of the same data.


source_id: 2605.15026
source: arxiv
title: SemaTune: Semantic-Aware Online OS Tuning with Large Language Models
authors: ["Georgios Liargkovas","Mihir Nitin Joshi","Hubertus Franke","Kostis Kaffes"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15026v1
tags: ["model infrastructure","reinforcement learning","retrieval augmented generation"]
org: IBM Research
fetched_at: 2026-05-16T01:26:31.197Z
classified_at: 2026-05-16T01:26:31.165Z
abstract:
Online OS tuning can improve long-running services, but existing controllers are poorly matched to live hosts. They treat scheduler, power, memory, and I/O controls as black-box variables and optimize a scalar reward. This view ignores cross-knob policy structure, breaks down when application metrics are unavailable, and can send a running service into degraded regions that persist after the bad setting is removed. We present SemaTune, a host-side framework for steady-state OS tuning with bounded language-model guidance. SemaTune turns knob schemas, telemetry, current configuration, recent action--response history, and retrieved prior runs into a compact decision context. A fast loop proposes low-latency updates, a slower loop periodically revises the search strategy, and every proposed change passes through typed validation before reaching kernel or sysctl interfaces. This lets the controller reason about OS-control meaning and indirect performance signals while keeping model cost, latency, and authority constrained. We evaluate SemaTune on 13 live workloads from five benchmark suites while tuning up to 41 Linux parameters. Across the suite, SemaTune improves stable-phase performance by 72.5\% over default settings and by 153.3\% relative to the strongest non-LLM baseline. A 30-window session costs about \$0.20 in model calls. With only host-level metrics, SemaTune still outperforms baselines given direct application objectives by 93.7 percentage points, while avoiding severe degraded regions reached by structure-blind exploration.


source_id: 2605.15000
source: arxiv
title: Quantifying and Mitigating Premature Closure in Frontier LLMs
authors: ["Rebecca Handler","Suhana Bedi","Nigam Shah"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15000v1
tags: ["evaluation","safety and security","alignment"]
org: Stanford University
fetched_at: 2026-05-16T01:28:03.681Z
classified_at: 2026-05-16T01:28:03.650Z
abstract:
Premature closure, or committing to a conclusion before sufficient information is available, is a recognized contributor to diagnostic error but remains underexamined in large language models (LLMs). We define LLM premature closure as inappropriate commitment under uncertainty: providing an answer, recommendation, or clinical guidance when the safer response would be clarification, abstention, escalation, or refusal. We evaluated five frontier LLMs across structured and open-ended medical tasks. In MedQA (n = 500) and AfriMed-QA (n = 490) questions where the correct choice had been removed, models still selected an answer at high rates, with baseline false-action rates of 55-81% and 53-82%, respectively. In open-ended evaluation, models gave inappropriate answers on an average of 30% of 861 HealthBench questions and 78% of 191 physician-authored adversarial queries. Safety-oriented prompting reduced premature closure across models, but residual failure persisted, highlighting the need to evaluate whether medical LLMs know when not to answer.


source_id: 2605.15015
source: arxiv
title: Small, Private Language Models as Teammates for Educational Assessment Design
authors: ["Chris Davis Jaldi","Anmol Saini","Shan Zhang","Noah Schroeder","Cogan Shimizu","Eleni Ilkou"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15015v1
tags: ["evaluation","model infrastructure","safety and security"]
org: Wright State University
fetched_at: 2026-05-16T01:29:27.254Z
classified_at: 2026-05-16T01:29:27.220Z
abstract:
Generative AI increasingly supports educational design tasks, e.g., through Large Language Models (LLMs), demonstrating the capability to design assessment questions that are aligned with pedagogical frameworks (e.g., Bloom's taxonomy). However, they often rely on subjective or limited evaluation methods; focus primarily on proprietary models; or rarely systematically examine generation, evaluation, or deployment constraints in real educational settings. Meanwhile, Small Language Models (SLMs) have emerged as local alternatives that better address privacy and resource limitations; yet their effectiveness for assessment tasks remains underexplored. To address this gap, we systematically compare LLMs and SLMs for assessment question design; evaluate generation quality across Bloom's taxonomy levels using reproducible, pedagogically grounded metrics; and further assess model-based judging against expert-informed evaluation by analyzing reliability and agreement patterns. Results show that SLMs achieve competitive performance across key pedagogically motivated quality dimensions while enabling local, privacy-sensitive deployment. However, model-based evaluations also exhibit systematic inconsistencies and bias relative to expert ratings. These findings provide evidence to posit language models as bounded assistants in assessment workflows; underscore the necessity of Human-in-the-Loop; and advance the automated educational question generation field by examining quality, reliability, and deployment-aware trade-offs.


source_id: 2605.15012
source: arxiv
title: Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance
authors: ["Kai Yan","Alexander G. Schwing","Yu-Xiong Wang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15012v1
tags: ["reinforcement learning","training","evaluation"]
org: University of Illinois Urbana-Champaign
fetched_at: 2026-05-16T01:31:18.015Z
classified_at: 2026-05-16T01:31:17.982Z
abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has achieved great success in developing Large Language Models (LLMs) with chain-of-thought rollouts for many tasks such as math and coding. Nevertheless, RLVR struggles with sample efficiency on difficult problems where correct rollouts are hard to generate. Prior works propose to address this issue via demonstration-guided RLVR, i.e., to conduct Supervised FineTuning (SFT) when RL fails; however, SFT often requires a lot of data, which can be expensive to acquire. In this paper, we propose FEST, a FEw-ShoT demonstration-guided RLVR algorithm. It attains compelling results with only 128 demonstrations randomly selected from an SFT dataset. We find that three components are vital for the success: supervised signal, on-policy signal, and decaying weights on the few-shot SFT dataset to prevent overfitting from multiple-epoch training. On several benchmarks, FEST outperforms baselines with magnitudes less SFT data, even matching their performance with full dataset.


source_id: 2605.14999
source: arxiv
authors: ["Pooja Prajod","Hannes Cools","Thomas Röggla","Pablo Cesar","Abdallah El Ali"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14999v1
tags: ["governance","evaluation","monitoring"]
org: Centrum Wiskunde & Informatica Amsterdam
fetched_at: 2026-05-16T01:32:48.795Z
classified_at: 2026-05-16T01:32:48.765Z
title:
Towards Gaze-Informed AI Disclosure Interfaces: Eye-Tracking Attentional and Cognitive Load While Reading AI-Assisted News

abstract:
As generative AI becomes increasingly integrated into journalism, designing effective AI-use disclosures that inform readers without imposing unnecessary burden is a key challenge. While prior research has primarily focused on trust and credibility, the impact of disclosures on readers' attentional and cognitive load remains underexplored. To address this gap, we conducted a $3\times2\times2$ mixed factorial study manipulating the level of AI-use disclosure detail (none, one-line, detailed), news type (politics, lifestyle), and role of AI (editing, partial content generation), measuring load via NASA-TLX and eye-tracking. Our results reveal a significant attentional cost: one-line disclosures resulted in significantly higher fixation durations and saccade counts, particularly for AI-edited content. Detailed disclosures did not impose additional burden. Drawing on Information-Gap Theory, we argue that brief labels may trigger increased visual scrutiny by alerting readers to AI use without providing enough information. NASA-TLX scores and pupil diameter showed no significant differences across conditions, suggesting that AI-use disclosures do not impose cognitive burden regardless of the detail level. Interview insights contextualize these findings and reveal a strong preference for detailed or ``detail-on-demand'' designs. Our findings inform the design of gaze-informed adaptive disclosure interfaces that dynamically adjust transparency levels based on readers' attentional patterns and news context.


source_id: 2605.15183
source: arxiv
title: When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15183v1
tags: ["interpretability","theories and proofs","training"]
org: Stanford University
fetched_at: 2026-05-16T01:34:05.125Z
classified_at: 2026-05-16T01:34:05.088Z
authors:
["ML Nissen Gonzalez","Melwina Albuquerque","Laurence Wroe","Jacob Meyer Cohen","Logan Riggs Smith","Thomas Dooms"]

abstract:
Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation.


source_id: 2605.15196
source: arxiv
title: RefDecoder: Enhancing Visual Generation with Conditional Video Decoding
authors: ["Xiang Fan","Yuheng Wang","Bohan Fang","Zhongzheng Ren","Ranjay Krishna"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15196v1
tags: ["model infrastructure","evaluation","training"]
org: University of Washington
fetched_at: 2026-05-16T01:35:07.789Z
classified_at: 2026-05-16T01:35:07.760Z
abstract:
Video generation powers a vast array of downstream applications. However, while the de facto standard, i.e., latent diffusion models, typically employ heavily conditioned denoising networks, their decoders often remain unconditional. We observe that this architectural asymmetry leads to significant loss of detail and inconsistency relative to the input image. To address this, we argue that the decoder requires equal conditioning to preserve structural integrity. We introduce RefDecoder, a reference-conditioned video VAE decoder by injecting high-fidelity reference image signal directly into the decoding process via reference attention. Specifically, a lightweight image encoder maps the reference frame into the detail-rich high-dimensional tokens, which are co-processed with the denoised video latent tokens at each decoder up-sampling stage. We demonstrate consistent improvements across several distinct decoder backbones (e.g., Wan 2.1 and VideoVAE+), achieving up to +2.1dB PSNR over the unconditional baselines on the Inter4K, WebVid, and Large Motion reconstruction benchmarks. Notably, RefDecoder can be directly swapped into existing video generation systems without additional fine-tuning, and we report across-the-board improvements in subject consistency, background consistency, and overall quality scores on the VBench I2V benchmark. Beyond I2V, RefDecoder generalizes well to a wide range of visual generation tasks such as style transfer and video editing refinement.


source_id: 2605.15138
source: arxiv
title: Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
authors: ["Saisab Sadhu","Pratinav Seth","Vinay Kumar Sankarapu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15138v1
tags: ["machine unlearning","model infrastructure","evaluation"]
org: Saisab Sadhu, Pratinav Seth, Vinay Kumar Sankarapu Lexsi Labs saisab.sadhu@lexsi.ai
fetched_at: 2026-05-16T01:36:38.751Z
classified_at: 2026-05-16T01:36:38.719Z
abstract:
Standard unlearning evaluations measure behavioral suppression in full precision, immediately after training, despite every deployed language model being quantized first. Recent work has shown that 4-bit post-training quantization can reverse machine unlearning; we show this is not a tuning artefact but a systematic dual failure: gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model. Both failures trace to the same root cause: across all baselines, per-parameter updates lie 47-828x below the NF4 quantization bin width; updates diffused across billions of parameters cannot clear quantization bin boundaries, a consequence we formalize as a sparsity-permanence tradeoff. We present MANSU (Mechanistic-Aligned Null-Space Unlearning), which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph, circuit-restricted null-space projection with a diagonal-Fisher retain bound, and a per-parameter magnitude floor guaranteeing quantization survival by construction. We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression, a distinction existing metrics cannot make. Across multiple model families and hazard benchmarks, MANSU is the first method to jointly satisfy all four properties with margin on each (meaningful forgetting, retain preservation, non-positive PTQ gap, and structural erasure), while gradient-based baselines recover up to +0.05 accuracy under compression.


source_id: 2605.15157
source: arxiv
title: Hand-in-the-Loop: Improving Dexterous VLA via Seamless Interventional Correction
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15157v1
tags: ["control","evaluation","training"]
org: Meta Robotics Institute
fetched_at: 2026-05-16T01:38:07.143Z
classified_at: 2026-05-16T01:38:07.107Z
authors:
["Zhuohang Li","Liqun Huang","Wei Xu","Zhengming Zhu","Nie Lin","Xiao Ma","Xinjun Sheng","Ruoshi Wen"]

abstract:
Vision-Language-Action (VLA) models are prone to compounding errors in dexterous manipulation, where high-dimensional action spaces and contact-rich dynamics amplify small policy deviations over long horizons. While Interactive Imitation Learning (IIL) can refine policies through human takeover data, applying it to high-degree-of-freedom (DoF) robotic hands remains challenging due to a command mismatch between human teleoperation and policy execution at the takeover moment, which causes abrupt robot-hand configuration changes, or "gesture jumps". We present Hand-in-the-Loop (HandITL), a seamless human-in-the-loop intervention method that blends human corrective intent with autonomous policy execution to avoid gesture jumps during bimanual dexterous manipulation. Compared with direct teleoperation takeover, HandITL reduces takeover jitter by 99.8% and preserves robust post-takeover manipulation, reducing grasp failures by 87.5% and mean completion time by 19.1%. We validate HandITL on tasks requiring bimanual coordination, tool use, and fine-grained long-horizon manipulation. When used to collect intervention data for policy refinement, HandITL yields policies that outperform those trained with standard teleoperation data by 19% on average across three long-horizon dexterous tasks.


source_id: 2605.15154
source: arxiv
title: RoSHAP: A Distributional Framework and Robust Metric for Stable Feature Attribution
authors: ["Lanxin Xiang","Liang Shi","Youhui Ye","Boyu Jiang","Dawei Zhou","Feng Guo"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15154v1
tags: ["interpretability","theories and proofs","evaluation"]
org: Virginia Tech
fetched_at: 2026-05-16T01:39:24.066Z
classified_at: 2026-05-16T01:39:24.035Z
abstract:
Feature attribution analysis is critical for interpreting machine learning models and supporting reliable data-driven decisions. However, feature attribution measures often exhibit stochastic variation: different train--test splits, random seeds, or model-fitting procedures can produce substantially different attribution values and feature rankings. This paper proposes a framework for incorporating stochastic nature of feature attribution and a robust attribution metric, RoSHAP, for stable feature ranking based on the SHAP metric. The proposed framework models the distribution of feature attribution scores and estimates it through bootstrap resampling and kernel density estimation. We show that, under mild regularity conditions, the aggregated feature attribution score is asymptotically Gaussian, which greatly reduces the computational cost of distribution estimation. The RoSHAP summarizes the distribution of SHAP into a robust feature-ranking criterion that simultaneously rewards features that are active, strong, and stable. Through simulations and real-data experiments, the proposed framework and RoSHAP outperform standard single-run attribution measures in identifying signal features. In addition, models built using RoSHAP-selected features achieve predictive performance comparable to full-feature models while using substantially fewer predictors. The proposed RoSHAP approach improves the stability and interpretability of machine learning models, enabling reliable and consistent insights for analysis.


source_id: 2605.15134
source: arxiv
title: Training ML Models with Predictable Failures
authors: ["Will Schwarzer","Scott Niekum"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15134v1
tags: ["evaluation","training","reinforcement learning"]
org: University of Massachusetts Amherst
fetched_at: 2026-05-16T01:40:55.529Z
classified_at: 2026-05-16T01:40:55.494Z
abstract:
Estimating how often an ML model will fail at deployment scale is central to pre-deployment safety assessment, but a feasible evaluation set is rarely large enough to observe the failures that matter. Jones et al. (2025) address this by extrapolating from the largest k failure scores in an evaluation set to predict deployment-scale failure rates. We give a finite-k decomposition of this estimator's forecast error and show that it has a built-in bias toward over-prediction in the typical case, which is the safety-favorable direction. This bias is offset when the evaluation set misses a rare high-failure mode that the deployment set contains, leaving the forecast to under-predict at deployment scale. We propose a fine-tuning objective, the forecastability loss, that addresses this failure mode. In two proof-of-concept experiments, a language-model password game and an RL gridworld, fine-tuning substantially reduces held-out forecast error while preserving primary-task capability and achieving safety similar to that of supervised baselines.


source_id: 2605.15133
source: arxiv
title: Causal Foundation Models with Continuous Treatments
authors: ["Christopher Stith","Medha Barath","Vahid Balazadeh","Jesse C. Cresswell","Rahul G. Krishnan"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15133v1
tags: ["training","evaluation","theories and proofs"]
org: Vector Institute
fetched_at: 2026-05-16T01:42:11.514Z
classified_at: 2026-05-16T01:42:11.475Z
abstract:
Causal inference, estimating causal effects from observational data, is a fundamental tool in many disciplines. Of particular importance across a variety of domains is the continuous treatment setting, where the variable of intervention has a continuous range. This setting is far less explored and represents a substantial shift from the binary treatment setting, with models needing to represent effects across a continuum of treatment values. In this paper, we present the first causal foundation model for the continuous treatment setting. Our model meta-learns the ability to predict causal effects across a wide variety of unseen tasks without additional training or fine-tuning. First, we design a novel prior over data-generating processes with continuous treatment variables in order to generate a rich causal training corpus. We then train a transformer to reconstruct individual treatment-response curves given only observational data, leveraging in-context learning to amortize expensive Bayesian posterior inference. Our model achieves state-of-the-art performance on individual treatment-response curve reconstruction tasks compared to causal models which are trained specifically for those tasks.


source_id: 2605.15131
source: arxiv
title: Natural Synthesis: Outperforming Reactive Synthesis Tools with Large Reasoning Models
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15131v1
tags: ["symbolic AI","evaluation","theories and proofs"]
org: CISPA Helmholtz Center for Information Security
fetched_at: 2026-05-16T01:43:44.606Z
classified_at: 2026-05-16T01:43:44.574Z
authors:
["Frederik Schmitt","Matthias Cosler","Niklas Metzger","Julian Siber","Vladimir Krsmanovic","Mohamed Ghanem","Bernd Finkbeiner"]

abstract:
Reactive synthesis, the problem of automatically constructing a hardware circuit from a logical specification, is a long-standing challenge in formal verification. It is elusive for two reasons: It is algorithmically hard, and writing formal specifications by hand is notoriously difficult. In this paper, we tackle both sides of the problem. For the algorithmic side, we present a neuro-symbolic approach to reactive synthesis that couples large reasoning models with model checkers to iteratively repair a synthesized Verilog implementation via sound symbolic feedback. Our approach solves more benchmarks than the best dedicated tools in the annual synthesis competition and extends to constructing parameterized systems, a problem known to be undecidable. On the specification side, we introduce an autoformalization step that shifts the specification task from temporal logic to natural language by introducing a hand-authored dataset of natural-language specifications for evaluation. We demonstrate performance comparable to that of starting from formal specifications, establishing natural synthesis as a viable end-to-end workflow.


source_id: 2605.15110
source: arxiv
title: Proposal and study of statistical features for string similarity computation and classification
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15110v1
tags: ["theories and proofs","safety and security","training"]
org: Universidade Tecnológica Federal do Paraná (UTFPR), Paraná, Brazil
fetched_at: 2026-05-16T01:45:14.724Z
classified_at: 2026-05-16T01:45:14.690Z
authors:
["E. O. Rodrigues","D. Casanova","M. Teixeira","V. Pegorini","F. Favarim","E. Clua","A. Conci","Panos Liatsis"]

abstract:
Adaptations of features commonly applied in the field of visual computing, co-occurrence matrix (COM) and run-length matrix (RLM), are proposed for the similarity computation of strings in general (words, phrases, codes and texts). The proposed features are not sensitive to language related information. These are purely statistical and can be used in any context with any language or grammatical structure. Other statistical measures that are commonly employed in the field such as longest common subsequence, maximal consecutive longest common subsequence, mutual information and edit distances are evaluated and compared. In the first synthetic set of experiments, the COM and RLM features outperform the remaining state-of-the-art statistical features. In 3 out of 4 cases, the RLM and COM features were statistically more significant than the second best group based on distances (P-value < 0.001). When it comes to a real text plagiarism dataset, the RLM features obtained the best results.


source_id: 2605.15113
source: arxiv
title: Learning from Language Feedback via Variational Policy Distillation
authors: ["Yang Li","Erik Nijkamp","Semih Yavuz","Shafiq Rayhan Joty"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15113v1
tags: ["reinforcement learning","training","evaluation"]
org: Salesforce AI Research
fetched_at: 2026-05-16T01:46:54.062Z
classified_at: 2026-05-16T01:46:54.028Z
abstract:
Reinforcement learning from verifiable rewards (RLVR) suffers from sparse outcome signals, creating severe exploration bottlenecks on complex reasoning tasks. Recent on-policy self-distillation methods attempt to address this by utilizing language feedback to generate dense, token-level supervision. However, these approaches rely on a fixed, passive teacher to interpret the feedback. As the student policy improves, the teacher's zero-shot assessment capabilities plateau, ultimately halting further learning. To overcome this, we propose Variational Policy Distillation (VPD), a framework that formalizes learning from language feedback as a Variational Expectation-Maximization (EM) problem. VPD co-evolves both policies: in the E-step, the teacher is actively refined on trajectory outcomes via an adaptive trust-region update, translating textual feedback into a dynamically improved target token distribution. In the M-step, the student internalizes this dense distributional guidance on its own on-policy rollouts. By continuously improving the teacher's ability to extract actionable signals from textual critique, VPD overcomes the limitations of passive distillation. Evaluated across diverse sources of diagnostic feedback on scientific reasoning and code generation tasks, VPD consistently outperforms both standard RLVR and existing self-distillation baselines. Finally, by stress-testing our framework on rigid mathematical reasoning and cold-start regimes, we illuminate the fundamental bounds of feedback-driven self-distillation compared to pure environment-driven RL.


source_id: 2605.15122
source: arxiv
title: CoCo-InEKF: State Estimation with Learned Contact Covariances in Dynamic, Contact-Rich Scenarios
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15122v1
tags: ["reinforcement learning","training","theories and proofs"]
org: ETH Zurich
fetched_at: 2026-05-16T01:48:24.694Z
classified_at: 2026-05-16T01:48:24.667Z
authors:
["Michael Baumgartner","David Müller","Agon Serifi","Ruben Grandia","Espen Knoop","Markus Gross","Moritz Bächer"]

abstract:
Robust state estimation for highly dynamic motion of legged robots remains challenging, especially in dynamic, contact-rich scenarios. Traditional approaches often rely on binary contact states that fail to capture the nuances of partial contact or directional slippage. This paper presents CoCo-InEKF, a differentiable invariant extended Kalman filter that utilizes continuous contact velocity covariances instead of binary contact states. These learned covariances allow the method to dynamically modulate contact confidence, accounting for more nuanced conditions ranging from firm contact to directional slippage or no contact. To predict these covariances for a set of predefined contact candidate points, we employ a lightweight neural network trained end-to-end using a state-error loss. This approach eliminates the need for heuristic ground-truth contact labels. In addition, we propose an automated contact candidate selection procedure and demonstrate that our method is insensitive to their exact placement. Experiments on a bipedal robot demonstrate a superior accuracy-efficiency tradeoff for linear velocity estimation, as well as improved filter consistency compared to baseline methods. This enables the robust execution of challenging motions, including dancing and complex ground interactions -- both in simulation and in the real world.


source_id: 2605.15085
source: arxiv
title: From Data to Action: Accelerating Refinery Optimization with AI
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15085v1
tags: ["monitoring","interpretability","evaluation"]
org: Stanford University
fetched_at: 2026-05-16T01:49:19.768Z
classified_at: 2026-05-16T01:49:19.735Z
authors:
["Dániel Pfeifer","Ábrahám Papp","Tibor Bernáth","Tamás Zoltán Varga","Márk Czifra","Botond Szilágyi","Edith Alice Kovács"]

abstract:
Nowadays refinery optimization utilizes sheer amounts of data, which can be handled with modern Linear Programming (LP) software, but the interpreting and applying the results remains challenging. Large petrochemical companies use massive models, with hundreds of thousands of input matrix elements. The LP solution is mathematically correct, but simplifications are made in the model, and data supply errors may occur. Therefore, further insight is needed to trust the results. The LP solver does not have a memory, so additional understanding could be gained by analyzing historical data and comparing it to the current plan. As such, machine learning approaches were suggested to support decision making based on the LP solution. Among these, Anomaly Detection tools are proposed to be used in tandem with the LP output. A transformed version of the popular ECOD methodology is applied. New methods are proposed to handle high-dimensional data: choosing the most informative pairs. Then, this is used alongside two 2D Anomaly Detection algorithms, revealing several business opportunities and data supply errors in the MOL refinery scheduling and planning architecture.


source_id: 2605.15079
source: arxiv
title: Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15079v1
tags: ["model infrastructure","evaluation","governance"]
org: Massachusetts Institute of Technology
fetched_at: 2026-05-16T01:51:04.637Z
classified_at: 2026-05-16T01:51:04.603Z
authors:
["Rafi Al Attrach","Rajna Fani","Sebastian Lobentanzer","Joan Giner-Miguelez","Debanshu Das","Varuni H. K.","Nobin Sarwar","Rajat Ghosh","Anwai Archit","Surbhi Motghare","Christina Conrad Parry","Luis Oala","Lara Grosso","Joaquin Vanschoren","Steffen Vogler","Sujata Goswami","Eric S. Rosenthal","Marzyeh Ghassemi","Matthew McDermott","Tom Pollard"]

abstract:
Croissant has emerged as the metadata standard for machine learning datasets, providing a structured, JSON-LD-based format that makes dataset discovery, automated ingestion, and reproducible analysis machine-checkable across ML platforms. Adoption has accelerated, and NeurIPS now requires Croissant metadata in every submission to its dataset tracks. Yet in practice Croissant generation usually starts with uploading data to a public platform, a path infeasible for governed and large local repositories that hold much of the high-value data ML increasingly relies on. We release Croissant Baker, a local-first, open-source command-line tool that generates validated Croissant metadata directly from a dataset directory through a modular handler registry. We evaluate Croissant Baker on over 140 datasets, scaling to MIMIC-IV at 886 million rows and 374 Parquet files. On held-out comparisons against producer-authored or standards-derived ground truth, Croissant Baker reaches 97-100% agreement across multiple domains.


source_id: 2605.15082
source: arxiv
authors: ["Libin Zhu","Damek Davis","Dmitriy Drusvyatskiy","Maryam Fazel"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15082v1
tags: ["theories and proofs","interpretability","evaluation"]
org: University of Washington
fetched_at: 2026-05-16T01:52:34.328Z
classified_at: 2026-05-16T01:52:34.298Z
title:
Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models

abstract:
We study a prototypical situation when a learned predictor can discover useful low-dimensional structure in data, while using fewer samples than are needed for accurate prediction. Specifically, we consider the problem of recovering a multi-index polynomial $f^*(x)=h(Ux)$, with $U\in\mathbb{R}^{r\times d}$ and $r\ll d$, from finitely many data/label pairs. Importantly, the target function depends on input $x$ only through the projection onto an unknown $r$-dimensional central subspace. The algorithm we analyze is appealingly simple: fit kernel ridge regression (KRR) to the data and compute the Average Gradient Outer Product (AGOP) from the fitted predictor. Our main results show that under reasonable assumptions the top $r$-dimensional eigenspace of AGOP provably recovers the central subspace, even in regimes when the prediction error remains large. Specifically, if the target function $f^*$ has degree $p^*$, it is known that $n\asymp d^{p^*}$ samples are necessary for KRR to achieve accurate prediction. In contrast, we show that if a low degree $p$ component of $f^*$ already carries all relevant directions for prediction, subspace recovery occurs in the much lower sample regime $n\asymp d^{p+δ}$ for any $δ\in(0,1)$. Our results thus demonstrate a separation between prediction and representation, and provide an explanation for why iterative kernel methods such as Recursive Feature Machines (RFM) can be sample-efficient in practice.


source_id: 2605.15055
source: arxiv
title: DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15055v1
tags: ["reinforcement learning","training","theories and proofs"]
org: Fudan University
fetched_at: 2026-05-16T01:53:53.779Z
classified_at: 2026-05-16T01:53:53.748Z
authors:
["Quanhao Li","Junqiu Yu","Kaixun Jiang","Yujie Wei","Zhen Xing","Pandeng Li","Ruihang Chu","Shiwei Zhang","Yu Liu","Zuxuan Wu"]

abstract:
Reinforcement learning has emerged as a powerful tool for improving diffusion-based text-to-image models, but existing methods are largely limited to single-task optimization. Extending RL to multiple tasks is challenging: joint optimization suffers from cross-task interference and imbalance, while cascade RL is cumbersome and prone to catastrophic forgetting. We propose DiffusionOPD, a new multi-task training paradigm for diffusion models based on Online Policy Distillation (OPD). DiffusionOPD first trains task-specific teachers independently, then distills their capabilities into a unified student along the student own rollout trajectories. This decouples single-task exploration from multi-task integration and avoids the optimization burden of solving all tasks jointly from scratch. Theoretically, we lift the OPD framework from discrete tokens to continuous-state Markov processes, deriving a closed-form per-step KL objective that unifies both stochastic SDE and deterministic ODE refinement via mean-matching. We formally and empirically demonstrate that this analytic gradient provides lower variance and better generality compared to conventional PPO-style policy gradients. Extensive experiments show that DiffusionOPD consistently surpasses both multi-reward RL and cascade RL baselines in training efficiency and final performance, while achieving state-of-the-art results on all evaluated benchmarks.


source_id: 2605.15050
source: arxiv
authors: ["Yuxin Guo","Dongrui Deng","Pulkit Grover"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15050v1
tags: ["interpretability","evaluation","theories and proofs"]
org: Carnegie Mellon University
fetched_at: 2026-05-16T01:55:13.933Z
classified_at: 2026-05-16T01:55:13.900Z
title:
Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems

abstract:
Recently, deep generative models have been used for posterior inference in inverse problems, including high-stakes applications in medical imaging and scientific discovery, where the uncertainty of a prediction can matter as much as the prediction itself. However, posterior uncertainty is difficult to interpret because it can mix ambiguity inherent to the forward operator with uncertainty propagated through inference. We introduce a structural decomposition of posterior uncertainty that isolates intrinsic ambiguity. A cascade formulation makes this ambiguity accessible for calibration analysis, enabling qualitative diagnostics and simulation-based calibration tests that reveal failure modes that remain hidden when models are selected by reconstruction quality alone. We first validate the approach on a Gaussian example with analytical posterior structure, then illustrate the decomposition on accelerated magnetic resonance imaging (MRI), and finally apply the calibration diagnostics to electroencephalography (EEG) source imaging.


source_id: 2605.15051
source: arxiv
title: An Interpretable Latency Model for Speculative Decoding in LLM Serving
authors: ["Linghao Kong","Megan Flynn","Michael Peng","Nir Shavit","Mark Kurtz","Alexandre Marques"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15051v1
tags: ["model infrastructure","evaluation","theories and proofs"]
org: Red Hat AI
fetched_at: 2026-05-16T01:56:41.451Z
classified_at: 2026-05-16T01:56:41.421Z
abstract:
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified by a larger target model in parallel. While prior work demonstrates substantial speedups in isolated or fixed-batch settings, the behavior of SD in production serving systems remains poorly understood: request load varies over time, and effective batch size emerges from the serving system rather than being directly controlled or observed. In this work, we develop a simple and interpretable latency model for SD in LLM serving. We infer effective batch size from request rate using Little's Law and decompose per-request demand into load-independent and load-dependent components for prefill, drafting, and verification. We validate our model using extensive measurements from vLLM across verifier and drafter model sizes, prefill and decode lengths, request rates, draft lengths, and acceptance probabilities. The model accurately describes observed latency, explains why speedups often diminish as server load increases, and characterizes how draft length, acceptance rate, and verifier-drafter size shape latency across serving conditions, with implications for configuring SD in deployed systems. We further show how the framework extends to mixture of experts models, where sparse expert activation changes the effective service costs across load regimes. Together, our results provide a structured framework for understanding SD in real LLM serving systems.


source_id: 2605.15009
source: arxiv
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15009v1
tags: ["training","evaluation","model infrastructure"]
org: University of California, Irvine
fetched_at: 2026-05-16T01:58:19.977Z
classified_at: 2026-05-16T01:58:19.941Z
title:
DeepTokenEEG Enhancing Mild Cognitive Impairment and Alzheimers Classification via Tokenized EEG Features

authors:
["Thinh Nguyen-Quang","Minh Long Ngo","Ngoc-Son Nguyen","Nguyen Thanh Vinh","Huy-Dung Han","Bui Thanh Tung","Nguyen Quang Linh","Khuong Vo","Manoj Vishwanath","Hung Cao"]

abstract:
The detection of Alzheimers disease (AD) is considered crucial, as timely intervention can improve patient outcomes. Electroencephalogram (EEG)-based diagnosis has been recognized as a non-invasive, accessible, and cost-effective approach for AD detection; however, it faces challenges related to data availability, accuracy of modern deep learning methods, and the time-consuming nature of expert-based interpretation. In this study, a novel lightweight and high-performance model, DeepTokenEEG, was designed for the diagnosis of AD and the classification of EEG signals from AD patients, individuals with other neurological conditions, and healthy subjects. Unlike traditional heavy-weight models, DeepTokenEEG ultilizes spatial and temporal tokenizer that effectively captures AD-related biomarkers in both temporal and frequency domain with only 0.29 million paramaters. Trained in a combined dataset of 274 subjects, including 180 AD cases, and 94 healthy controls, the proposed method achieves a maximum recorded accuracy of 100% on specific frequency bands, representing an improvement of 1.41-15.35% over state-of-the-art methods on the same dataset. These results indicate the potential of DeepTokenEEG for early detection and screening of AD, with promising applicability for deployment due to its compact size.


source_id: 2605.15032
source: arxiv
title: Multi-Block Attention for Efficient Channel Estimation in IRS-Assisted mmWave MIMO
authors: ["Mehrdad Momen-Tayefeh","Mehrshad Momen-Tayefeh","Maryam Sabbaghian"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15032v1
tags: ["evaluation","theories and proofs","training"]
org: University of Tehran
fetched_at: 2026-05-16T01:59:47.618Z
classified_at: 2026-05-16T01:59:47.587Z
abstract:
Intelligent Reflecting Surfaces (IRSs) are a promising technology for enhancing the spectral and energy efficiency of millimeter-wave (mmWave) multiple-input multiple-output (MIMO) systems. In these systems, accurate channel estimation remains challenging due to the passive nature of IRS elements and the high pilot overhead in large-scale deployments. This paper presents a deep learning-based Multi-Block Attention (MBA) framework for efficient cascaded channel estimation in IRS-assisted mmWave MIMO systems that utilize orthogonal frequency division multiplexing (OFDM). First, we show the optimality of the discrete Fourier transform (DFT) and Hadamard matrices as phase configurations for least squares (LS) estimation. To reduce training overhead, we selectively deactivate IRS elements and compensate for induced feature loss using a two-stage architecture: (i) a Convolutional Attention Network (CAN) for spatial correlation recovery and (ii) a Complex Multi-Convolutional Network (CMN) for noise suppression. The MBA architecture mitigates error propagation through attention-guided feature refinement and denoising. Simulation results indicate that the MBA method reduces pilot overhead by up to 87% compared to the LS estimator. Additionally, at signal-to-noise ratios of 10 dB, our proposed method achieves approximately 51% lower normalized mean squared error (NMSE) than leading methods. It also maintains low computational complexity and adapts effectively to various propagation environments.


source_id: 2605.15035
source: arxiv
title: TopoPrimer: The Missing Topological Context in Forecasting Models
authors: ["Zara Zetlin","Kayhan Moharreri","Maria Safi"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15035v1
tags: ["evaluation","theories and proofs","model infrastructure"]
org: Stanford University
fetched_at: 2026-05-16T02:01:31.197Z
classified_at: 2026-05-16T02:01:31.165Z
abstract:
We introduce TopoPrimer, a framework that makes the global topological structure of the series population an explicit input to any forecasting model. TopoPrimer improves accuracy across diverse domains, stabilizes forecasts under seasonal demand spikes, and closes the cold-start gap. Precomputed once per domain via persistent homology and spectral sheaf coordinates, TopoPrimer deploys per token for fully-trained models and as a lightweight adapter for pre-trained backbones. Of these two components, sheaf coordinates are the primary accuracy driver. Across four public benchmarks on Chronos and TimesFM, TopoPrimer consistently improves forecasting accuracy, with gains of up to 7.3% MSE on ECL. The topology advantage persists with near-identical magnitude across zero-shot and fine-tuned backbones, suggesting topology and per-series training capture complementary signals. The gains are most pronounced in difficult regimes. Under peak seasonal demand, classical and zero-shot models degrade by up to 50%, while TopoPrimer stays within 10%. At cold start with no item history, TopoPrimer reduces MAE by 27% over a topology-free baseline.


source_id: 2605.14981
source: arxiv
title: Distance-Matrix Wasserstein Statistics for Scalable Gromov--Wasserstein Learning
authors: ["Ao Xu","Tieru Wu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14981v1
tags: ["theories and proofs","information geometry","training"]
org: Jilin University
fetched_at: 2026-05-16T02:03:11.140Z
classified_at: 2026-05-16T02:03:11.103Z
abstract:
Gromov--Wasserstein (GW) distances compare graphs, shapes, and point clouds through internal distances, without requiring a common coordinate system. This invariance is powerful, but discrete GW is a nonconvex quadratic optimal transport problem and is difficult to estimate at scale. We propose \emph{Distance-Matrix Wasserstein} (DMW), a hierarchy of Wasserstein statistics comparing laws of random finite distance matrices. Rather than optimizing a global point-level alignment, DMW samples $n$ points from each space, records their pairwise distances, and transports the resulting matrix laws. We prove that DMW is a relaxation and lower bound of GW, and establish a reverse approximation inequality: the GW--DMW gap is controlled by the Wasserstein error of approximating each original measure with $n$ samples. Hence population DMW converges to GW as sampled subspaces become dense. We further give finite-sample bounds, including intrinsic-dimensional rates that depend on the data manifold rather than the ambient matrix dimension $\binom n2$. For scalable computation, we introduce sliced and multi-scale DMW; for $p=1$, the sliced multi-scale dissimilarity yields positive-definite exponential kernels. Experiments on synthetic metric spaces, scalability benchmarks, graph classification, and two-sample testing validate the theory and demonstrate an interpretable GW-style proxy for structural comparison.


source_id: 2605.14982
source: arxiv
title: Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
authors: ["Sanjeev Manivannan","Shuban V"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14982v1
tags: ["reinforcement learning","theories and proofs","training"]
org: Indian Institute of Technology Madras
fetched_at: 2026-05-16T02:04:43.049Z
classified_at: 2026-05-16T02:04:43.016Z
abstract:
We address the discounted reward setting in reinforcement learning (RL). To mitigate the value approximation challenges in policy gradient methods, actor-critic approaches have been developed and are known to converge to stationary points under suitable assumptions. However, these methods rely on first-order updates. In contrast, second-order optimization provides principled curvature-aware updates that are proven to accelerate convergence, but its application in RL is limited by the computational complexity of Hessian estimation. In this work, we analyze second-order approximations for the actor update that leverage the full curvature information of the objective as much as possible. A stable approximation requires treating the action-value function as locally constant with respect to policy parameters, which does not generally hold in policy gradient methods. We show that this approximation becomes well-justified under a two-timescale actor-critic framework, where the critic evolves on a faster timescale and can be treated as quasi-stationary during actor updates. Building on this insight, we formulate a second-order actor-critic method for the discounted reward setting that leverages Hessian-vector product (HVP) computations, resulting in a computationally efficient and stable second-order update.


source_id: 2605.14995
source: arxiv
title: Explainable Detection of Depression Status Shifts from User Digital Traces
authors: ["Loris Belcastro","Francesco Gervino","Fabrizio Marozzo","Domenico Talia","Paolo Trunfio"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14995v1
tags: ["interpretability","evaluation","monitoring"]
org: University of Calabria
fetched_at: 2026-05-16T02:06:08.928Z
classified_at: 2026-05-16T02:06:08.898Z
abstract:
Every day, users generate digital traces (e.g., social media posts, chats, and online interactions) that are inherently timestamped and may reflect aspects of their mental state. These traces can be organized into temporal trajectories that capture how a user's mental health signals evolve, including phases of improvement, deterioration, or stability. In this work, we propose an explainable framework for detecting and analyzing depression-related status shifts in user digital traces. The approach combines multiple BERT-based models to extract complementary signals across different dimensions (e.g., sentiment, emotion, and depression severity). Such signals are then aggregated over time to construct user-level trajectories that are analyzed to identify meaningful change points. To enhance interpretability, the framework integrates a large language model to generate concise and human-readable reports that describe the evolution of mental-health signals and highlight key transitions. We evaluate the framework on two social media datasets. Results show that the approach produces more coherent and informative summaries than direct LLM-based reporting, achieving higher coverage of user history, stronger temporal coherence, and improved sensitivity to change points. An ablation study confirms the contribution of each component, particularly temporal modeling and segmentation. Overall, the method provides an interpretable view of mental health signals over time, supporting research and decision making without aiming at clinical diagnosis.


source_id: 2605.15172
source: arxiv
title: MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs
authors: ["Rui Wen","Mark Russinovich","Andrew Paverd","Jun Sakuma","Ahmed Salem"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15172v1
tags: ["governance","evaluation","model infrastructure"]
org: Microsoft Azure
fetched_at: 2026-05-16T02:07:35.684Z
classified_at: 2026-05-16T02:07:35.653Z
abstract:
Backdoor attacks pose a serious security threat to large language models (LLMs), which are increasingly deployed as general-purpose assistants in safety- and privacy-critical applications. Existing LLM backdoors rely primarily on content-based triggers, requiring explicit modification of the input text. In this work, we show that this assumption is unnecessary and limiting. We introduce MetaBackdoor, a new class of backdoor attacks that exploits positional information as the trigger, without modifying textual content. Our key insight is that Transformer-based LLMs necessarily encode token positions to process ordered sequences. As a result, length-correlated positional structure is reflected in the model's internal computation and can be used as an effective non-content trigger signal. We demonstrate that even a simple length-based positional trigger is sufficient to activate stealthy backdoors. Unlike prior attacks, MetaBackdoor operates on visibly and semantically clean inputs and enables qualitatively new capabilities. We show that a backdoored LLM can be induced to disclose sensitive internal information, including proprietary system prompts, once a length condition is satisfied. We further demonstrate a self-activation scenario, where normal multi-turn interaction can move the conversation context into the trigger region and induce malicious tool-call behavior without attacker-supplied trigger text. In addition, MetaBackdoor is orthogonal to content-based backdoors and can be composed with them to create more precise and harder-to-detect activation conditions. Our results expand the threat model of LLM backdoors by revealing positional encoding as a previously overlooked attack surface. This challenges defenses that focus on detecting suspicious text and highlights the need for new defense strategies that explicitly account for positional triggers in modern LLM architectures.


source_id: 2605.14953
source: arxiv
title: Efficient Online Conformal Selection with Limited Feedback
authors: ["Sreenivas Gollapudi","Kostas Kollias","Kamesh Munagala","Ali Sinop"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14953v1
tags: ["theories and proofs","reinforcement learning","evaluation"]
org: Stanford University
fetched_at: 2026-05-16T02:08:55.425Z
classified_at: 2026-05-16T02:08:55.393Z
abstract:
We address the problem of conformal selection, where an agent must select a minimal subset of options to ensure that at least one ``success'' is identified with a pre-specified target probability $φ$. While traditional online conformal prediction focuses on maintaining validity for the observed sequence, minimizing the resource cost (efficiency) of such selections, especially under limited feedback, remains a significant challenge. In this work, we consider settings with the most limited ``bandit'' feedback, and demonstrate that the simple Adaptive Conformal Inference (ACI) update rule, when applied to the appropriate control parameter or dual variable, is both adversarially valid, ensuring the success target is met on average for any input sequence (and hence under distribution shifts), and stochastically efficient, achieving sublinear efficiency regret for $i.i.d.$ inputs against an appropriate stochastic benchmark. We show such guarantees under canonical models capturing bandit and semi-bandit feedback to the agent via a unifying algorithmic technique, and analytic framework involving Lyapunov functions. Our approach handles more complex settings than prior work, while requiring significantly less feedback, and our results provide a new theoretical bridge between efficient online learning with limited feedback and distribution-free uncertainty quantification.


source_id: 2605.14967
source: arxiv
title: InfoSFT: Learn More and Forget Less with Information-Aware Token Weighting
authors: ["Mahdi Sabbaghi","George Pappas","Adel Javanmard","Hamed Hassani"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14967v1
tags: ["training","evaluation","model infrastructure"]
org: University of Pennsylvania
fetched_at: 2026-05-16T02:10:39.023Z
classified_at: 2026-05-16T02:10:38.990Z
abstract:
Supervised fine-tuning (SFT) provides the standard approach for teaching LLMs new behaviors from offline expert demonstrations. However, standard SFT uniformly fits all samples -- including those with low likelihood under the base model -- which can disproportionately drive training updates toward overfitting specific samples rather than learning the target behavior. Moreover, adapting to these unlikely samples induces substantial policy shifts that degrade prior capabilities. Existing methods mitigate this by filtering, regenerating, or down-weighting low-likelihood data. In doing so, they often suppress precisely the novel behaviors the base model has yet to learn. We propose InfoSFT, a principled weighting scheme for the SFT objective that concentrates learning signals on maximally informative, medium-confidence tokens -- those neither overly familiar to the base model nor too unlikely to cause instability. Requiring only a one-line modification to the standard token-wise loss, InfoSFT demonstrably improves generalization over vanilla SFT and likelihood-weighted baselines across math, code, and chain-of-thought tasks with diverse model families, while better preserving pre-existing capabilities.


source_id: 2605.15118
source: arxiv
title: Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks
authors: ["Karthik Raghu Iyer","Yazdan Jamshidi","Nicholas Bray","Alexey A. Shvets"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15118v1
tags: ["governance","evaluation","model infrastructure"]
org: Palo Alto Networks
fetched_at: 2026-05-16T02:12:15.365Z
classified_at: 2026-05-16T02:12:15.333Z
abstract:
We introduce a reusable framework for auditing whether LLM attack benchmarks collectively cover the threat surface: a 4$\times$6 Target $\times$ Technique matrix grounded in STRIDE, constructed from a 507-leaf taxonomy -- 401 data-populated and 106 threat-model-derived leaves -- of inference-time attacks extracted from 932 arXiv security studies (2023--2026). The matrix enables benchmark-external validation -- auditing collective coverage rather than individual benchmark consistency. Applying it to six public benchmarks reveals that the three primary frameworks (HarmBench, InjecAgent, AgentDojo) occupy non-overlapping cells covering at most 25\% of the matrix, while entire STRIDE threat categories (Service Disruption, Model Internals) lack any standardized evaluation, despite published attacks in these categories achieving 46$\times$ token amplification and 96\% attack success rates through mechanisms which no benchmark tests. The corpus of 2,521 unique attack groups further reveals pervasive naming fragmentation (up to 29 surface forms for a single attack) and heavy concentration in Safety \& Alignment Bypass, structural properties invisible at smaller scale. The taxonomy, attack records, and coverage mappings are released as extensible artifacts; as new benchmarks emerge, they can be mapped onto the same matrix, enabling the community to track whether evaluation gaps are closing.


source_id: 2605.15097
source: arxiv
authors: ["Xinran Zheng","Alfredo Pesoli","Marco Valleri","Suman Jana","Lorenzo Cavallaro"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15097v1
tags: ["evaluation","model infrastructure","governance"]
org: University College London
fetched_at: 2026-05-16T02:13:40.699Z
classified_at: 2026-05-16T02:13:40.660Z
title:
Veritas: A Semantically Grounded Agentic Framework for Memory Corruption Vulnerability Detection in Binaries

abstract:
Detecting memory corruption vulnerabilities in stripped binaries requires recovering object semantics, interprocedural propagation, and feasible triggers from low-level, lossy representations. Recent LLM-based approaches improve code understanding, but reliable detection still requires grounding in memory-relevant semantics and runtime feasibility evidence. We present Veritas, a semantically grounded framework for binary memory corruption vulnerability detection. Veritas combines a static slicer over RetDec-lifted LLVM IR, a dual-view LLM detector that reasons step by step over grounded flows using decompiled C and selective LLVM IR, and a multi-agent validator that checks hypotheses against debugger-visible artifacts and runtime evidence. The slicer reconstructs value-flow relations from LLVM-IR facts, including def-use, calls, returns, globals, and pointer operations, and emits compact witness-backed flow objects. The detector uses these artifacts to reason about control flow, bounds, and object correspondence without rediscovering whole-binary propagation. The validator confirms or rejects candidates through guided debugging, breakpoint inspection, and memory-checking oracles. We implement Veritas as a modular pipeline and evaluate it on a curated benchmark of real-world binary vulnerability cases. Veritas achieves 90\% recall. For false-positive assessment, we exhaustively validate and manually verify 623 detector candidates and audit additional candidates from larger cases. The exhaustive subset produces no false positives, while the additional audit identifies two confirmed false positives. In a real-world application, Veritas discovered a previously unknown Apple vulnerability that was confirmed and assigned a CVE. These results support semantic grounding as an operational design principle for practical binary vulnerability detection.


source_id: 2605.15084
source: arxiv
title: PickleFuzzer: A Case Study in Fuzzing for Discrepancies Between Python Pickle Implementations
authors: ["Justin Applegate","Andreas Kellas"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15084v1
tags: ["safety and security","monitoring","model infrastructure"]
org: Stanford University
fetched_at: 2026-05-16T02:15:07.304Z
classified_at: 2026-05-16T02:15:07.276Z
abstract:
Python's native serialization protocol, pickle, is a powerful but insecure format for transferring untrusted data. It is frequently used, especially for saving machine learning models, despite known security challenges. While developers sometimes mitigate this risk by restricting imports during unpickling or using static and dynamic analysis tools, these approaches are error-prone and depend heavily on accurate interpretations of the Pickle Virtual Machine (PVM) opcodes. Discrepancies across Python's three native PVM modules can lead to incorrect detection of malicious payloads and undermine existing defenses. To efficiently and scalably identify discrepancies, we present PickleFuzzer, a custom generation-based fuzzer that identifies inconsistencies across pickle implementations. PickleFuzzer generates pickle objects, passes them to each implementation, and detects differences in thrown exceptions or changes to key internal states. It generates pickle objects using a grammar, which we developed to account for the missing pickle specification. It determines discrepancies by comparing the execution behaviors of each test implementation, rather than requiring a specification-derived oracle. PickleFuzzer detected 14 new discrepancies between the pickle implementations. Four discrepancies are critical and can be used to bypass security-critical scanning tools like those deployed on the popular model hosting platform, Hugging Face. We disclosed all findings to the Python Software Foundation for remediation, and additionally disclosed the security issues to a bug bounty platform and were awarded a $750 bounty. We demonstrate that differential testing is a viable approach for identifying security-relevant discrepancies in important pickle implementations, and our work can lead to promising future directions for finding deeper pickle bugs with more directed fuzzing.


source_id: 2605.14932
source: arxiv
title: Toward Securing AI Agents Like Operating Systems
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14932v1
tags: ["governance","evaluation","model infrastructure"]
org: Max Planck Institute for Security and Privacy, Germany
fetched_at: 2026-05-16T02:16:47.249Z
classified_at: 2026-05-16T02:16:47.213Z
authors:
["Lukas Pirch","Micha Horlboge","Patrick Großmann","Syeda Mahnur Asif","Klim Kireev","Thorsten Holz","Konrad Rieck"]

abstract:
Autonomous agents based on large language models (LLMs) are rapidly emerging as a general-purpose technology, with recent systems such as OpenClaw extending their capabilities through broad tool use, third-party skills, and deeper integration into user environments. At the same time, these agentic systems introduce substantial security risks by combining unconstrained capabilities with access to sensitive user data. In this work, we investigate the security of LLM-based agents through the lens of operating systems. We argue that both face strikingly similar challenges in isolating resources, separating privileges, and mediating communication. Guided by this perspective, we survey the current landscape of open-source agents, derive a unified agent architecture, and systematically analyze potential attack vectors. To validate this analysis, we conduct a case study evaluating four widely used OpenClaw-like agents. Even under modest attacker capabilities, we find that several protection mechanisms fail in practice and that secure operation requires detailed system knowledge and careful configuration. However, we also observe that while some agentic capabilities remain insecure by design, many vulnerabilities can be mitigated using well-established techniques from operating system security. We conclude with a set of recommendations for the secure design of agentic systems.


source_id: 2605.15047
source: arxiv
title: Analyzing Codes of Conduct for Online Safety in Video Games at Scale
authors: ["Jiuming Jiang","Shidong Pan","Daniel W Woods","Jingjie Li"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.15047v1
tags: ["governance","evaluation","model infrastructure"]
org: University of Edinburgh
fetched_at: 2026-05-16T02:18:08.504Z
classified_at: 2026-05-16T02:18:08.474Z
abstract:
Online video games have become major online social spaces where users interact, compete, and create together. These spaces, however, expose users to a wide spectrum of online harms, including harassment, discrimination, inappropriate content, privacy breach, cheating, and more. The shape and severity of such harms vary across game design, mechanics, and community context. To mitigate these harms, game companies issue Codes of Conduct (CoCs) that articulate online safety rules and direct players to safety resources. However, it remains unclear how prevalent CoCs are, what safety, security and privacy violations they govern, and whether they meet growing regulatory and industry expectations. We develop and leverage CONDUCTIFY, a pipeline for identifying and analyzing CoCs at scale. Applied to Steam, the largest PC game marketplace, it located the available CoCs for 350 of the 9,586 multiplayer titles on Steam. We found that CoCs are more available among popular, adult-oriented, and community-driven games, while most multiplayer games operate without CoCs despite regulatory and industry recommendations. Although over 80% of the games with CoCs available consistently address traditional security and safety violations, their governance approaches vary substantially across types of violations. A further asymmetry emerges in specificity. Compared with harms related to gameplay mechanics, the articulations of interpersonal harm and the underage player safety are often less specific, despite their relevance to many game communities. Together, these results inform the improvement of online safety governance and CoC enforcement practices, and building better safety infrastructure for the community of players and developers.


source_id: 2605.14859
source: arxiv
title: Do Coding Agents Understand Least-Privilege Authorization?
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14859v1
tags: ["safety and security","agent foundations","evaluation"]
org: evolvent.co/en/research/authbench
fetched_at: 2026-05-16T02:19:26.369Z
classified_at: 2026-05-16T02:19:26.343Z
authors:
["Zheng Yan","Jingxiang Weng","Charles Chen","Dengyun Peng","Ethan Qin","Jiannan Guan","Jinhao Liu","Qiming Yu","Yixin Yuan","Fanqing Meng","Carl Che","Mengkang Hu"]

abstract:
As coding agents gain access to shells, repositories, and user files, least-privilege authorization becomes a prerequisite for safe deployment: an agent should receive enough authority to complete the task, without unnecessary authority that exposes sensitive surfaces.To study whether current models can infer this boundary themselves, we first introduce permission-boundary inference, where a model maps a task instruction and terminal environment to a file-level read/write/execute policy, and AuthBench, a benchmark of 120 realistic terminal tasks with human-reviewed permission labels and executable validators for utility and attack outcomes.AuthBench shows that authorization is not a simple conservative-versus-permissive calibration problem: frontier models often omit permissions required by the execution chain while also granting unused or sensitive accesses.Increasing inference-time reasoning does not resolve this mismatch. Instead, each model moves toward a model-specific authorization attractor: more reasoning makes it more consistent in its own failure mode, whether broad-but-exposed or tight-but-brittle.This suggests that direct policy generation is the bottleneck, because a single generation must both discover all necessary accesses and reject all unnecessary ones.We therefore propose Sufficiency-Tightness Decomposition, which first generates a coverage-oriented policy by forward-simulating the task and then audits each granted entry for grounding and sensitivity.Across tested models, this decomposition improves sensitive-task success by up to 15.8% on tightness-biased models while reducing attack success across all evaluated models.


source_id: 2605.14750
source: arxiv
title: EVA: Editing for Versatile Alignment against Jailbreaks
authors: ["Yi Wang","Hongye Qiu","Yue Xu","Sibei Yang","Zhan Qin","Minlie Huang","Wenjie Wang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14750v1
tags: ["governance","evaluation","model infrastructure"]
org: Stanford University
fetched_at: 2026-05-16T02:20:50.007Z
classified_at: 2026-05-16T02:20:49.976Z
abstract:
Large Language Models (LLMs) and Vision Language Models (VLMs) have demonstrated impressive capabilities but remain vulnerable to jailbreaking attacks, where adversaries exploit textual or visual triggers to bypass safety guardrails. Recent defenses typically rely on safety fine-tuning or external filters to reduce the model's likelihood of producing harmful content. While effective to some extent, these methods often incur significant computational overheads and suffer from the safety utility trade-off, degrading the model's performance on benign tasks. To address these challenges, we propose EVA (Editing for Versatile Alignment against Jailbreaks), a novel framework that pioneers the application of direct model editing for safety alignment. EVA reframes safety alignment as a precise knowledge correction task. Instead of retraining massive parameters, EVA identifies and surgically edits specific neurons responsible for the model's susceptibility to harmful instructions, while leaving the vast majority of the model unchanged. By localizing the updates, EVA effectively neutralizes harmful behaviors without compromising the model's general reasoning capabilities. Extensive experiments demonstrate that EVA outperforms baselines in mitigating jailbreaks across both LLMs and VLMs, offering a precise and efficient solution for post-deployment safety alignment.


source_id: 2605.14786
source: arxiv
title: Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces
authors: ["William Lugoloobi","Samuelle Marro","Jabez Magomere","Joss Wright","Chris Russell"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14786v1
tags: ["governance","evaluation","model infrastructure"]
org: Oxford Internet Institute, University of Oxford
fetched_at: 2026-05-16T02:22:11.477Z
classified_at: 2026-05-16T02:22:11.445Z
abstract:
As LLM-based agents increasingly browse the web on users' behalf, a natural question arises: can websites passively identify which underlying model powers an agent? Doing so would represent a significant security risk, enabling targeted attacks tailored to known model vulnerabilities. Across 14 frontier LLMs and four web environments spanning information retrieval and shopping tasks, we show that an agent's actions and interaction timings, captured via a passive JavaScript tracker, are sufficient to identify the underlying model with up to 96\% F1. We formalise this attack surface by demonstrating that classifiers trained on agent actions generalise across model sizes and families. We further show that strong classifiers can be trained from few interaction traces and that agent identity can be inferred early within an episode. Injecting randomised timing delays between actions substantially degrades classifier performance, but does not provide robust protection: a classifier retrained on delayed traces largely recovers performance. We release our harness and a labelled corpus of agent traces \href{https://github.com/KabakaWilliam/known_actions}{here}.


source_id: 2605.14799
source: arxiv
title: Can Visual Mamba Improve AI-Generated Image Detection? An In-Depth Investigation
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14799v1
tags: ["safety and security","evaluation","model infrastructure"]
org: Stanford University
fetched_at: 2026-05-16T02:23:11.698Z
classified_at: 2026-05-16T02:23:11.673Z
authors:
["Mamadou Keita","Wassim Hamidouche","Hessen Bougueffa Eutamene","Abdelmalik Taleb-Ahmed","Xianxun Zhu","Abdenour Hadid"]

abstract:
In recent years, computer vision has witnessed remarkable progress, fueled by the development of innovative architectures such as Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs), diffusion-based architectures, Vision Transformers (ViTs), and, more recently, Vision-Language Models (VLMs). This progress has undeniably contributed to creating increasingly realistic and diverse visual content. However, such advancements in image generation also raise concerns about potential misuse in areas such as misinformation, identity theft, and threats to privacy and security. In parallel, Mamba-based architectures have emerged as versatile tools for a range of image analysis tasks, including classification, segmentation, medical imaging, object detection, and image restoration, in this rapidly evolving field. However, their potential for identifying AI-generated images remains relatively unexplored compared to established techniques. This study provides a systematic evaluation and comparative analysis of Vision Mamba models for AI-generated image detection. We benchmark multiple Vision Mamba variants against representative CNNs, ViTs, and VLM-based detectors across diverse datasets and synthetic image sources, focusing on key metrics such as accuracy, efficiency, and generalizability across diverse image types and generative models. Through this comprehensive analysis, we aim to elucidate Vision Mamba's strengths and limitations relative to established methodologies in terms of applicability, accuracy, and efficiency in detecting AI-generated images. Overall, our findings highlight both the promise and current limitations of Vision Mamba as a component in systems designed to distinguish authentic from AI-generated visual content. This research is crucial for enhancing detection in an age where distinguishing between real and AI-generated content is a major challenge.


source_id: 2605.14605
source: arxiv
title: One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries
authors: ["Itay Zloczower","Eyal Lenga","Gilad Gressel","Yisroel Mirsky"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14605v1
tags: ["governance","evaluation","training"]
org: Ben-Gurion University of the Negev
fetched_at: 2026-05-16T02:24:38.670Z
classified_at: 2026-05-16T02:24:38.636Z
abstract:
Model providers increasingly release open weights or allow users to fine-tune foundation models through APIs. Although these models are safety-aligned before release, their safeguards can often be removed by fine-tuning on harmful data. Recent defenses aim to make models robust to such malicious fine-tuning, but they are largely evaluated only against fixed attacks that do not account for the defense. We show that these robustness claims are incomplete. Surveying 15 recent defenses, we identify several defense mechanisms and show that they share a single weakness: they obscure or misdirect the path to harmful behavior without removing the behavior itself. We then develop a unified adaptive attack that breaks defenses across all defense mechanisms. Our results show that current approaches do not provide robust security; they mainly stop the attacks they were designed against. We hope that our unified adaptive adversary for this domain will help future researchers and practitioners stress-test new defenses before deployment.


source_id: 2605.14718
source: arxiv
title: Adapting AlphaEvolve to Optimize Fully Homomorphic Encryption on TPUs
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14718v1
tags: ["evaluation","theories and proofs","model infrastructure"]
org: Georgia Institute of Technology
fetched_at: 2026-05-16T02:26:03.616Z
classified_at: 2026-05-16T02:26:03.587Z
authors:
["Shruthi Gorantala","Jianming Tong","Asra Ali","Baiyu Li","Jonathan Katz","Jeremy Kun","Thomas Steinke","Abhradeep Thakurta","Julian Walker","Amir Yazdanbakhsh"]

abstract:
The deployment of Fully Homomorphic Encryption (FHE) at scale is hindered due to its heavy computational overhead. While specialized hardware accelerators like Google Tensor Processing Units (TPUs) can help, mapping complex cryptographic kernels onto such architectures remains a challenge. Efficient execution requires co-optimization between the systolic array-based Matrix Multiplication Unit (MXU) and Vector Processing Units (VPUs), as well as the orchestration of data movement across the vector register files. Existing compiler stacks often abstract low-level hardware utilization, requiring developers to adopt a manual trial-and-error process that often results in fragmented execution and underutilized resources. To accelerate this development process, we use AlphaEvolve to automate the exploration of hardware-aware cryptographic-kernel optimizations. We frame optimization as an evolutionary search problem, utilizing the closed-loop system provided by AlphaEvolve, that leverages LLM-driven code generation. We use real-world feedback from hardware execution and rigorous correctness testing to guide the evolution process. We evaluate AlphaEvolve optimization on primitives for both the TFHE (Jaxite) and CKKS (CROSS) FHE schemes on Google Cloud TPUv5e, a contemporary TPU architecture. Within 24 hours of automated exploration, AlphaEvolve discovered implementation-level optimizations that improve TFHE bootstrap latency by 2.5x and CKKS rotation and multiplication latency by 1.31x and 1.18x, respectively, relative to human-engineered state of the art. These results demonstrate that AlphaEvolve can be used to enable researchers to navigate the optimization trade-offs between cryptography, compilers, and hardware accelerators.


source_id: 2605.14633
source: arxiv
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14633v1
tags: ["safety and security","monitoring","theories and proofs"]
org: Wuhan University
fetched_at: 2026-05-16T02:27:40.548Z
classified_at: 2026-05-16T02:27:40.519Z
title:
Capacitive Touchscreens at Risk: A Practical Side-Channel Attack on Smartphones via Electromagnetic Emanations

authors:
["Yukun Cheng","Changhai Ou","Shiyu Zhu","Jinyuan Zhang","Zhenfang Qiu","Xingshuo Han","Tianwei Zhang","Yuan Li","Shihui Zheng"]

abstract:
Capacitive touchscreens in modern smartphones introduce severe side-channel vulnerabilities. However, existing attacks often require restrictive conditions or invasive measurements. This paper presents TESLA, a novel, contactless electromagnetic (EM) side-channel attack that exploits inherent EM emanations during touchscreen scanning. We demonstrate that these emanations encode the spatiotemporal evolution of touch interactions, forming a unified leakage basis. By secretly placing an EM probe near the victim's device, TESLA enables attackers to extract highly sensitive information, including screen-unlocking PIN codes, keyboard inputs, interacting application categories, and continuous handwriting trajectories. Compared to existing attacks, TESLA offers a broader range of attack targets, more efficient sample acquisition, and operations in practical attack scenarios. Extensive evaluations on popular commercial smartphones, specifically the iPhone X, Xiaomi 10 Pro, Samsung S10, and Huawei Mate 30 Pro, validate the effectiveness of TESLA. It achieves remarkable inference accuracy in diverse settings such as private meeting rooms and public libraries, with success rates of 99.3% for PIN code recognition, 97.6% for keyboard input reconstruction, and 95.0% for application inference, respectively. Simultaneously, it attains a 76.8% character recognition accuracy and a high geometric similarity (Jaccard index of 0.74) for 2D handwriting trajectory reconstruction.


source_id: 2605.14591
source: arxiv
title: Privacy Auditing with Zero (0) Training Run
authors: ["Tudor Cebere","Mathieu Even","Linus Bleistein","Aurélien Bellet"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14591v1
tags: ["governance","evaluation","theories and proofs"]
org: EPFL
fetched_at: 2026-05-16T02:29:02.199Z
classified_at: 2026-05-16T02:29:02.172Z
abstract:
Privacy auditing provides empirical lower bounds on the differential privacy parameters of learning algorithms. Existing methods, however, require interventional access to the training pipeline, either to retrain multiple times or to randomize data inclusion. This is often infeasible for large deployed systems such as foundation models. We introduce Zero-Run privacy auditing, a post-hoc framework for auditing models using two fixed datasets: examples known to be training-set members and examples known to be non-members. In this observational regime, membership is no longer randomized; instead, member and non-member data often differ in distribution, so membership inference scores may reflect a distribution shift rather than algorithmic leakage. Drawing on ideas from causal inference, we formalize this confounding effect and propose two complementary corrections that yield valid privacy audits. Our first approach models the combined effect of distribution shift and algorithmic leakage as an adaptive composition, producing conservative global corrections. Our second approach conditions on observed data and adjusts pointwise membership guesses, yielding sharper instance-dependent bounds. Experiments on synthetic data and large-scale models show that Zero-Run auditing enables practical privacy evaluation when retraining or controlled data insertion is infeasible.


source_id: 2605.14587
source: arxiv
authors: ["Oubo Ma","Ruixiao Lin","Yang Dai","Jiahao Chen","Chunyi Zhou","Linkang Du","Shouling Ji"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14587v1
tags: ["reinforcement learning","safety and security","evaluation"]
org: Zhejiang University
fetched_at: 2026-05-16T02:30:38.699Z
classified_at: 2026-05-16T02:30:38.670Z
title:
Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning

abstract:
Extensive research has highlighted the severe threats posed by backdoor attacks to deep reinforcement learning (DRL). However, prior studies primarily focus on vanilla scenarios, while plasticity interventions have emerged as indispensable built-in components of modern DRL agents. Despite their effectiveness in mitigating plasticity loss, the impact of these interventions on DRL backdoor vulnerabilities remains underexplored, and this lack of systematic investigation poses risks in practical DRL deployments. To bridge this gap, we empirically study 14,664 cases integrating representative interventions and attack scenarios. We find that only one intervention (i.e., SAM) exacerbates backdoor threats, while other interventions mitigate them. Pathological analysis identifies that the exacerbation is attributed to backdoor gradient amplification, while the mitigation stems from activation pathway disruption and representation space compression. From these findings, we derive two novel insights: (1) a conceptual framework SCC for robust backdoor injection that deconstructs the mechanistic interplay between interventions and backdoors in DRL, and (2) abnormal loss landscape sharpness as a key indicator for DRL backdoor detection.


source_id: 2605.14514
source: arxiv
title: Defenses at Odds: Measuring and Explaining Defense Conflicts in Large Language Models
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14514v1
tags: ["safety and security","training","theories and proofs"]
org: Stanford University
fetched_at: 2026-05-16T02:31:59.353Z
classified_at: 2026-05-16T02:31:59.326Z
authors:
["Xiangtao Meng","Wenyu Chen","Chuanchao Zang","Xinyu Gao","Jianing Wang","Li Wang","Zheng Li","Shanqing Guo"]

abstract:
Large Language Models (LLMs) deployed in high-stakes applications must simultaneously manage multiple risks, yet existing defenses are almost exclusively evaluated in isolation under a one-shot deployment assumption. In practice, providers patch models incrementally throughout their lifecycle-responding to newly exposed vulnerabilities or targeted data-removal requests without retraining from scratch. This raises a fundamental but underexplored question: does a later defense preserve the protections established by an earlier one? We present the first systematic study of cross-defense interactions under sequential deployment. Evaluating 144 ordered sequences across three risk dimensions and three model families, we find that 38.9% exhibit measurable risk exacerbation on the originally defended dimension. These interactions are highly asymmetric and order-dependent. To explain these phenomena, we conduct a mechanistic analysis on representative deployment sequences. Using layer-wise representational divergence and activation patching, we localize each defense to a compact set of critical layers. In conflicting sequences, the overlapping critical layers exhibit strongly anti-aligned parameter updates, whereas benign orderings maintain near-orthogonal updates. PCA trajectory analysis reveals that defense collapse stems from activation pattern reversals in these shared layers. We further introduce a layer-wise conflict score that quantifies the geometric tension between defense-induced activation subspaces, offering mechanistic insight into the observed reversals. Guided by this diagnosis, we propose conflict-guided layer freezing, a lightweight mitigation that selectively freezes high-conflict layers during sequential deployment, preserving prior protections without degrading secondary defense performance.


source_id: 2605.14460
source: arxiv
title: Exploiting LLM Agent Supply Chains via Payload-less Skills
authors: ["Xinyu Liu","Yukai Zhao","Xing Hu","Xin Xia"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14460v1
tags: ["safety and security","agent foundations","evaluation"]
org: Zhejiang University
fetched_at: 2026-05-16T02:33:26.234Z
classified_at: 2026-05-16T02:33:26.195Z
abstract:
Autonomous agents powered by Large Language Models (LLMs) acquire external functionalities through third-party skills available in open marketplaces. Adopting these integrations broadens the potential attack surface, prompting a need for systematic security evaluation. Current auditing mechanisms are effective at identifying explicit code payloads and predefined threat contents through security scanning. These detection mechanisms are bypassed if malicious behaviors lack direct injection and are instead synthesized dynamically at runtime through the agent's inherent generative capabilities. Exploring this blind spot, we introduce Semantic Compliance Hijacking (SCH), a payload-less supply chain attack targeting autonomous coding environments. The SCH approach translates malicious goals into unstructured natural language instructions formatted as necessary compliance rules, leading the agent to generate and execute unauthorized code. To assess the real-world viability of this attack, we developed an automated pipeline to evaluate its effectiveness across a test matrix comprising three mainstream agent frameworks and three distinct foundation models using contextualized scenarios. The findings demonstrate the pervasive nature of this threat, with SCH achieving peak success rates of up to 77.67% for confidentiality breaches and 67.33% for Remote Code Execution (RCE) under the most vulnerable configurations. Furthermore, the introduction of Multi-Skill Automated Optimization (MS-AO) further boosted attack efficacy. By omitting recognizable Abstract Syntax Tree (AST) signatures and explicit harmful intents, the manipulated skill files maintained a 0.00% detection rate, evading current scanning tools. This research highlights an underexplored attack surface within agent supply chains, pointing to a necessary transition from signature-based detection models toward semantic intent validation.


source_id: 2605.14454
source: arxiv
title: LiSA: Lifelong Safety Adaptation via Conservative Policy Induction
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14454v1
tags: ["safety and security","agent foundations","monitoring"]
org: Google Cloud AI Research
fetched_at: 2026-05-16T02:34:57.859Z
classified_at: 2026-05-16T02:34:57.829Z
authors:
["Minbeom Kim","Lesly Miculicich","Bhavana Dalvi Mishra","Mihir Parmar","Phillip Wallis","Bharath Chandrasekhar","Kyomin Jung","Tomas Pfister","Long T. Le"]

abstract:
As AI agents move from chat interfaces to systems that read private data, call tools, and execute multi-step workflows, guardrails become a last line of defense against concrete deployment harms. In these settings, guardrail failures are no longer merely answer-quality errors: they can leak secrets, authorize unsafe actions, or block legitimate work. The hardest failures are often contextual: whether an action is acceptable depends on local privacy norms, organizational policies, and user expectations that resist pre-deployment specification. This creates a practical gap: guardrails must adapt to their own operating environments, yet deployment feedback is typically limited to sparse, noisy user-reported failures, and repeated fine-tuning is often impractical. To address this gap, we propose LiSA (Lifelong Safety Adaptation), a conservative policy induction framework that improves a fixed base guardrail through structured memory. LiSA converts occasional failures into reusable policy abstractions so that sparse reports can generalize beyond individual cases, adds conflict-aware local rules to prevent overgeneralization in mixed-label contexts, and applies evidence-aware confidence gating via a posterior lower bound, so that memory reuse scales with accumulated evidence rather than empirical accuracy alone. Across PrivacyLens+, ConFaide+, and AgentHarm, LiSA consistently outperforms strong memory-based baselines under sparse feedback, remains robust under noisy user feedback even at 20% label-flip rates, and pushes the latency--performance frontier beyond backbone model scaling. Ultimately, LiSA offers a practical path to secure AI agents against the unpredictable long tail of real-world edge risks.


source_id: 2605.14435
source: arxiv
title: Geographic Patterns in I2P Peer Selection: An Empirical Network Topology Analysis
authors: ["Siddique Abubakr Muntaka","Jess Kropczynski","Jacques Bou Abdo","Murat Ozer"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14435v1
tags: ["safety and security","theories and proofs","monitoring"]
org: University of Cincinnati
fetched_at: 2026-05-16T02:36:36.821Z
classified_at: 2026-05-16T02:36:36.792Z
abstract:
The Invisible Internet Project (I2P) routes data via encrypted, decentralized tunnels. Peer selection can significantly affect security and performance. This empirical study examines whether geographic location systematically influences I2P's routing topology. Consistent with I2P's design principles, which include avoiding multiple peers from the same /16 IP subnet to maximize anonymity, we conducted assortativity analysis, community detection, and permutation testing on data from 327 routers and 254 connections (SWARM-I2P). We found a network-level absence of significant geographic homophily. The assortativity coefficient was r = 0.017 (p = 0.222). Same-country connections (11.1%) are statistically near random expectation (10.91%). Community detection found 110 highly modular groups (Q = 0.972) only moderately aligned geographically (NMI = 0.521). We conclude that aggregate peer selection in I2P leads to a highly heterogeneous, random geographical mixing, providing a foundation for understanding the performance-anonymity tradeoff.


source_id: 2605.14418
source: arxiv
title: The Great Pretender: A Stochasticity Problem in LLM Jailbreak
authors: ["Jean-Philippe Monteuuis","Cong Chen","Jonathan Petit"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14418v1
tags: ["evaluation","safety and security","training"]
org: Microsoft Research
fetched_at: 2026-05-16T02:38:06.541Z
classified_at: 2026-05-16T02:38:06.510Z
abstract:
"Oh-Oh, yes, I'm the great pretender. Pretending that I'm doing well. My need is such, I pretend too much..." summarizes the state in the area of jailbreak creation and evaluation. You find this method to generate adversarial attacks proposed by a reputable institution (e.g., BoN from Anthropic or Crescendo from Microsoft Research). However, this method does not deliver on the promise claimed in the paper despite having top ASR scores against industry-grade LLMs. You successfully generate the jailbreak prompts against your target (open) model. However, the generated jailbreak prompt works against the target model with a 50% consecutive success rate (5 out of 10 attempts) despite having an 80% ASR (on paper) on the latest closed-source model (with a guardrail system)! This observation leads us to think. First, Attack Success Rate (ASR), the primary metric for LLM jailbreak benchmarking, is not a stable quantity. Second, published ASR numbers are therefore systematically inflated and incomparable across papers. Therefore, we wonder "Why a successful jailbreak prompt does not perform consistently well against a target model on which the prompts have been optimized?". To answer this question, we study the impact of stochasticity not only during attack evaluation but also during attack generation. Our evaluation includes several jailbreak attacks, models (different sizes and providers), and judges. In addition, we propose a new metric and two new frameworks (CAS-eval and CAS-gen). Our evaluation framework, CAS-eval, shows that an attack can have an ASR drop of up to 30 percentage points when a jailbreak prompt needs to succeed on more than one attempt. Thankfully, our attack generation framework (CAS-gen) improves previous jailbreak methods and helps them recover this loss of 30 percentage points!


source_id: 2605.14421
source: arxiv
title: MemLineage: Lineage-Guided Enforcement for LLM Agent Memory
authors: ["Ciyan Ouyang","Rui Hou"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14421v1
tags: ["safety and security","model infrastructure","theories and proofs"]
fetched_at: 2026-05-16T02:39:33.496Z
classified_at: 2026-05-16T02:39:33.465Z
abstract:
We introduce MemLineage, a defense for LLM agent memory that attaches both cryptographic provenance and LLM-mediated derivation lineage to every entry. Recent and concurrent work shows that untrusted content can be written into persistent agent state and re-enter later sessions as an instruction; the remaining systems question is how to preserve useful memory recall while preventing such state from justifying sensitive actions. MemLineage treats this as a chain-of-custody problem rather than a filtering problem. It is a six-module design around an RFC-6962 Merkle log over per-principal Ed25519-signed entries: a weighted derivation DAG records which retrieved entries influenced each new memory, and a max-of-strong-edges propagation rule makes Untrusted-Path Persistence hold for any chain whose attribution edges remain above threshold. The sensitive-action gate then refuses dispatches whose active justification descends from an external ancestor, while still allowing benign recall. We evaluate three defense cells against three memory-poisoning workloads on a deterministic mechanism-isolation harness; MemLineage is the only configuration in that harness that drives all three columns to zero ASR, while sub-millisecond per-operation overhead keeps it well below the noise floor of any LLM call. A Codex-backed AgentDojo bridge further separates strong-model behavior from defense-layer behavior: under an intentionally vulnerable tool-output profile, no-defense and signature-only baselines fail on all six banking pairs, while all MemLineage rows reduce strict AgentDojo ASR to zero. The core deterministic artifacts are byte-equal CI-verified; hosted-model AgentDojo and live-model sweeps are recorded as auditable logs rather than byte-pinned artifacts.

org:
State Key Laboratory of Cyberspace Security Defense Institute of Information Engineering, CAS Beijing, China


source_id: 2605.14431
source: arxiv
title: FuzzAgent: Multi-Agent System for Evolutionary Library Fuzzing
authors: ["Yunlong Lyu","Peng Chen","Fengyi Wu","Junzhe Yu","Kit Long Hon","Hao Chen"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14431v1
tags: ["safety and security","agent foundations","evaluation"]
org: The University of Hong Kong
fetched_at: 2026-05-16T02:41:02.546Z
classified_at: 2026-05-16T02:41:02.516Z
abstract:
Library fuzzing is essential for hardening the software supply chain, but adopting it at scale remains expensive. Practitioners still spend substantial effort on environment setup, struggle to generate harnesses that respect intricate API constraints, and lack reliable means to tell genuine library bugs from harness-induced crashes. Recent LLM-based systems automate parts of this pipeline, yet they typically operate as one-shot code generators that ignore runtime feedback, which limits both the depth of code they reach and the validity of the bugs they report. We argue that effective library fuzzing is iterative by nature: each campaign exposes new coverage bottlenecks and crashes, and the next campaign should evolve from these signals rather than restart from scratch. Building on this insight, we present FuzzAgent, a multi-agent system that turns library fuzzing into an evolutionary process, in which a team of specialized agents collaborates over the full fuzzing lifecycle and grounds every decision in concrete runtime evidence, so that the harness suite is successively refined toward deeper coverage and higher-fidelity crash analysis across rounds. We evaluate FuzzAgent on 20 real-world C/C++ libraries against four state-of-the-art baselines (OSS-Fuzz, OSS-Fuzz-Gen, PromptFuzz, and PromeFuzz). FuzzAgent completes the full fuzzing lifecycle for all 20 libraries without human intervention and reaches 179619 branches, exceeding OSS-Fuzz, PromptFuzz, PromeFuzz, and OSS-Fuzz-Gen by 45.1%, 73.2%, 92.1%, and 191.2%, respectively. FuzzAgent also identifies 102 genuine library bugs, 78 of which have already been acknowledged and fixed by upstream maintainers.


source_id: 2605.14396
source: arxiv
title: Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14396v1
tags: ["safety and security","training","evaluation"]
org: Purdue University
fetched_at: 2026-05-16T02:42:25.074Z
classified_at: 2026-05-16T02:42:25.044Z
authors:
["Chenyi Wang","Ruoyu Song","Raymond Muller","Jean-Philippe Monteuuis","Jonathan Petit","Z. Berkay Celik","Ryan Gerdes","Ming F. Li"]

abstract:
Autonomous vehicles depend on online HD map construction to perceive lane boundaries, dividers, and pedestrian crossings -- safety-critical road elements that directly govern motion planning. While existing pixel perturbation attacks can disrupt the mapping, they can be neutralized by standard adversarial defenses. We present MIRAGE, a framework for systematic discovery of semantic attacks that bypass adversarial defenses and degrade mapping predictions by finding plausible environmental variation (e.g. shadows, wet roads). MIRAGE exploits the latent manifold of real-world data learned by diffusion models, and searches for semantically mutated scenes neighboring the ground truth with the same road topology yet mislead the mapping predictions. We evaluate MIRAGE on nuScenes and demonstrate two attacks: (1) boundary removal, suppressing 57.7% of detections and corrupting 96% of planned trajectories; and (2) boundary injection, the only method that successfully injects fictitious boundaries, while pixel PGD and AdvPatch fail entirely. Both attacks remain potent under various adversarial defenses. We use two independent VLM judges to quantify realism, where MIRAGE passes as realistic 80--84% of the time (vs. 97--99% for clean nuScenes), while AdvPatch only 0--9%. Our findings expose a categorical gap in current adversarial defenses: semantic-level perturbations that manifest as legitimate environmental variation are substantially harder to mitigate than pixel-level perturbations.


source_id: 2605.14325
source: arxiv
title: Toward Covert Quantum Computing
authors: ["Evan J. D. Anderson","Kaushik Datta","Boulat A. Bash"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14325v1
tags: ["safety and security","theories and proofs","monitoring"]
org: Nobelis, Inc.
fetched_at: 2026-05-16T02:43:57.901Z
classified_at: 2026-05-16T02:43:57.872Z
abstract:
As quantum computers become available through multi-tenant cloud platforms, ensuring privacy against adversaries sharing the same quantum processing unit becomes critical. We introduce and explore \emph{covert quantum computing}, a new concept that ensures an adversary with access to all other quantum computational units (QCUs) of a quantum computer cannot detect computation on the subset that they cannot access. Analogous to covert communication, we employ information theory. However, since here the adversary controls the systems used for detection, we require a richer framework for covertness analysis that accounts for the use of quantum memories and adaptive operations. Thus, we adopt the \emph{quantum-strategy} framework used in quantum game theory and memory channel discrimination. Current quantum computers use planar graph circuit layouts and typically assume nearest-neighbor crosstalk. We derive discrete isoperimetric inequalities to show that, for an $n$-qubit circuit under this model, only $\mathcal{O}(\sqrt{n})$ border qubits provide detection information to the adversary. We then explore this scaling law on IQM's 54-qubit \emph{Emerald} processor and IBM's 156-qubit \emph{ibm\_fez} machine employing the Heron 2 architecture. We implement Ramsey experiments on qubits not used in computation, and detect nearest-neighbor crosstalk, as expected. However, we also observe long-range coupling effects beyond the border qubits, revealing a side channel that the adversary can exploit. We hypothesize that this long-range crosstalk is induced by leakage from the drive and control lines. Beyond weakening covertness, it exposes co-tenants to both adversarial and unintended crosstalk and degrades circuits that span spatially distributed qubits, motivating further work on spatial isolation and crosstalk characterization.


source_id: 2605.14387
source: arxiv
title: Model Forensics in AI-Native Wireless Networks: Taxonomy, Applications, and Case Study
authors: ["Pengyu Chen","Weiyang Li","Jin Xu","Jiacheng Wang","Ning Wang","Dusit Niyato","Tao Xiang"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14387v1
tags: ["safety and security","monitoring","model infrastructure"]
org: IEEE
fetched_at: 2026-05-16T02:45:39.507Z
classified_at: 2026-05-16T02:45:39.469Z
abstract:
As artificial intelligence (AI) is increasingly embedded in wireless networks, models are becoming core components that influence signal processing, resource scheduling and network control. However, model anomalies, tampering and malicious functions also introduce new security risks. In this article, we focus on model forensics in AI-native wireless networks. Specifically, we first discuss key problems including model authenticity verification, malicious function identification and accountability tracing, and summarize the main categories of model forensics. We then explain the role of model forensics in AI-native wireless networks and review representative application scenarios. In the case study, we use RF fingerprinting as an example and present two concrete workflows based on watermark authentication and backdoor detection, illustrating how provenance authentication and malicious behavior identification can be implemented in practice. The results show that model forensics can provide important support for anomaly assessment, provenance tracing and trustworthy operation in AI-native wireless networks. Finally, we outline several promising directions for future research in this emerging area.


source_id: 2605.14290
source: arxiv
title: Web Agents Should Adopt the Plan-Then-Execute Paradigm
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14290v1
tags: ["safety and security","model infrastructure","agent foundations"]
org: UC Berkeley
fetched_at: 2026-05-16T02:47:10.402Z
classified_at: 2026-05-16T02:47:10.371Z
authors:
["Julien Piet","Annabella Chow","Yiwei Hou","Muxi Lyu","Sylvie Venuto","Jinhao Zhu","Raluca Ada Popa","David Wagner"]

abstract:
ReAct has become the default architecture across LLM agents, and many existing web agents follow this paradigm. We argue that it is the wrong default for web agents. Instead, web agents should default to plan-then-execute: commit to a task-specific program before observing runtime web content, then execute it. The reason is that web content mixes inputs from many parties. An e-commerce product page may combine a seller's listing, customer reviews and sponsored advertisements. Under ReAct, all of this content flows into the model when deciding on the next action, creating a direct path for prompt injections to steer the agent's control flow. Plan-then-execute changes this boundary: untrusted data may influence values or branches inside a predefined execution graph, but it cannot redefine the user task or cause the model to synthesize new actions at runtime. We analyze WebArena, a popular web agent benchmark, and find that all tasks are compatible with plan-then-execute, while 80% can be completed with a purely programmatic plan, without any runtime LLM subroutine. We identify the main barrier to adopting plan-then-execute on the web: For it to work well, tools must map cleanly to semantic actions, with effects known before execution, so agents have enough information to plan. The web does not naturally expose that interface. Browser tools such as click, type, and scroll have page-dependent meanings. Planning at this layer is near-sighted: the agent can only see actions on the current page, and later actions appear only after it acts. Closing this gap requires typed interfaces that turn website interactions from clicks and keystrokes to task-level operations. This is an infrastructure problem, not a modeling problem. Web tasks do not need reactivity by default; they need typed, complete, auditable website APIs.


source_id: 2605.14289
source: arxiv
title: MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
authors: ["Weisen Jiang","Shuhao Chen","Sinno Jialin Pan"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14289v1
tags: ["model infrastructure","training","safety and security"]
org: Stanford University
fetched_at: 2026-05-16T02:48:43.284Z
classified_at: 2026-05-16T02:48:43.251Z
abstract:
Mixture-of-Experts (MoE) models scale capacity by combining specialized experts, but most existing approaches assume centralized access to training data. In practice, data are distributed across clients and cannot be shared due to privacy constraints, making unified MoE training challenging. We propose MetaMoE, a privacy-preserving framework that unifies independently trained, domain-specialized experts into a single MoE using public proxy data as surrogates for inaccessible private data. Central to MetaMoE is diversity-aware proxy selection, which selects client-domain-relevant and diverse samples from public data to effectively approximate private data distributions and supervise router learning. These proxies are further used to align expert training, improving expert coordination at unification time, while a context-aware router enhances expert selection across heterogeneous inputs. Experiments on computer vision and natural language processing benchmarks demonstrate that MetaMoE consistently outperforms recent privacy-preserving MoE unification methods. Code is available at https://github.com/ws-jiang/MetaMoE.


source_id: 2605.14291
source: arxiv
authors: ["Chengshuai Zhao","Zhen Tan","Dawei Li","Zhiyuan Yu","Huan Liu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14291v1
tags: ["safety and security","training","theories and proofs"]
org: Texas A&M University
fetched_at: 2026-05-16T02:50:28.721Z
classified_at: 2026-05-16T02:50:28.682Z
title:
To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model

abstract:
The rapid advancement of Large Vision-Language Models (LVLMs) is increasingly accompanied by unauthorized scraping and training on multimodal web data, posing severe copyright and privacy risks to data owners. Existing countermeasures, such as machine unlearning and watermarks, are inherent post-hoc approaches that act only after intellectual property infringement has already occurred. In this work, we propose MMGuard to empower data owners to proactively protect their multimodal data against unauthorized LVLM fine-tuning. MMGuard generates unlearnable examples by injecting human-imperceptible perturbations that actively exploit the learning dynamics of LVLMs. By minimizing the training loss, the perturbation creates an optimization shortcut, causing the model to overfit to the noise and thereby degrading downstream performance when the perturbation is absent during inference. To further strengthen this defense, MMGuard introduces a cross-modal binding disruption, strategically shifting LVLM attention to enforce a spurious correlation between the noise and the training target with theoretical guarantees. Enhanced by an ensemble learning strategy for cross-model transferability, MMGuard is evaluated against nine open-source LVLMs across six datasets. Our comprehensive results demonstrate effective, stealthy, and robust protection under white-box, gray-box, and black-box threat models, establishing a mechanistic advantage in proactively defending against aggressive fine-tuning exploitation.


source_id: 2605.14230
source: arxiv
title: On the (non-)resilience of encrypted controllers to covert attacks
authors: ["Philipp Binfet","Janis Adamek","Moritz Schulze Darup"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14230v1
tags: ["safety and security","control","theories and proofs"]
org: Stanford University
fetched_at: 2026-05-16T02:52:07.157Z
classified_at: 2026-05-16T02:52:07.122Z
abstract:
The security of networked control systems (NCS) is receiving increasing attention from both cyber-security and system-theoretic perspectives. The former focuses on classical IT security goals such as confidentiality, integrity, and availability of process data, while the latter investigates tailored attacks (and detection schemes), including covert and zero-dynamics attacks. Confidentiality in control systems can, for instance, be achieved by securely outsourcing the evaluation of the controller to third-party platforms, such as cloud services. The underlying technology enabling such secure computation often is homomorphic encryption (HE). Recent works in encrypted control have proposed modifications to underlying HE schemes to achieve not only confidentiality but also resilience to certain types of integrity attacks. While extensions in this direction are desirable in principle, we show that the integrity problem in encrypted control cannot be solved by public-key HE schemes alone due to their inherent malleability. In other words, the same homomorphisms that enable encrypted control % in the first place can be leveraged not only constructively but also destructively. More precisely, we demonstrate that NCS are vulnerable to covert attacks, even when encrypted control is employed. Remarkably, this remains possible without knowledge of an unencrypted model. Yet, resilience to such attacks can still be achieved through complementary techniques. We present an approach based on verifiable computation that integrates with modern homomorphic cryptosystems and is asymptotically secure while incurring no communication overhead.


source_id: 2605.14283
source: arxiv
title: Watermarking Game-Playing Agents in Perfect-Information Extensive-Form Games
authors: ["Juho Kim","Fei Fang","Tuomas Sandholm"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14283v1
tags: ["safety and security","theories and proofs","reinforcement learning"]
org: Carnegie Mellon University
fetched_at: 2026-05-16T02:53:34.184Z
classified_at: 2026-05-16T02:53:34.153Z
abstract:
Watermarking techniques for large language models (LLMs), which encode hidden information in the output so its source can be verified, have gained significant attention in recent days, thanks to their potential capability to detect accidental or deliberate misuse. Similar challenges involving model misuse also exist in the context of game-playing, such as when detecting the unauthorized use of AI tools in gaming platforms (e.g., cheating in online chess). In this paper, we initiate the study of how game-playing strategies can be watermarked. We show how the KGW watermark for LLMs can be adapted to watermark game-playing agents in perfect-information extensive-form games. The watermark can then be detected using a statistical test. We show that the degradation in the quality of the watermarked strategy profile, quantified by the expected utility, can be bounded, but there is a tradeoff between detectability and quality. In our experiments, we bootstrap the watermarking framework to various chess engines and demonstrate that a) the impact of the watermark on the quality of the strategy is negligible and b) the watermark can be detected with just a handful of games.


source_id: 2605.14917
source: arxiv
title: A Mutual Information Lower Bound for Multimodal Regression Active Learning
authors: ["Leonardo Ferreira Guilhoto","Akshat Kaushal","Paris Perdikaris"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14917v1
tags: ["theories and proofs","training","evaluation"]
org: University of Pennsylvania
fetched_at: 2026-05-16T02:54:42.885Z
classified_at: 2026-05-16T02:54:42.858Z
abstract:
Active learning for continuous regression has lacked an acquisition function that targets epistemic uncertainty when the predictive distribution is multimodal: variance misses modal disagreement, and information-theoretic targets like BALD are designed for discrete outputs. We introduce a Two-Index framework that makes this separation explicit: one stochastic index selects among competing model hypotheses (epistemic source), while a second governs within-hypothesis randomness (aleatoric source). An entropy decomposition within the framework identifies the mutual information between the output and the epistemic index as a principled acquisition objective, and we prove this quantity vanishes as the model is trained on growing datasets, confirming that it captures exactly the uncertainty data can resolve. Because this mutual information is intractable for continuous outputs, we derive the Mutual Information Lower Bound (MI-LB) acquisition function, a closed-form approximation for Mixture Density Network ensembles. On benchmarks featuring multimodal systems, MI-LB matches or beats every baseline evaluated and is the only method to do so consistently -- geometric and Fisher-based baselines compete only when the input space already encodes the multimodality, and collapse otherwise.


source_id: 2605.14663
source: arxiv
authors: ["Sebastian Kassing","Thomas Kruse"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14663v1
tags: ["theories and proofs","reinforcement learning","evaluation"]
org: Stanford University
fetched_at: 2026-05-16T02:56:13.926Z
classified_at: 2026-05-16T02:56:13.894Z
title:
Optimal Asymptotic Rates for (Stochastic) Gradient Descent under the Local PL-Condition: A Geometric Approach

abstract:
Stochastic gradient descent (SGD) has been studied extensively over the past decades due to its simplicity and broad applicability in machine learning. In this work, we analyze the local behavior of gradient descent and stochastic gradient descent for minimizing $C^2$-functions that satisfy the Polyak-Lojasiewicz (PL) inequality and under a multiplicative gradient noise model motivated by overparameterized neural networks. Using a geometric interpretation of the PL-condition, we prove a simple yet surprising fact: in this possibly non-convex setting, the asymptotic convergence rate of (S)GD matches the rate obtained for strongly convex quadratics.


source_id: 2605.14840
source: arxiv
title: In-Context Learning for Data-Driven Censored Inventory Control
authors: ["Sohom Mukherjee","Anh-Duy Pham","Richard Pibernik","Yunbei Xu"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14840v1
tags: ["theories and proofs","reinforcement learning","model infrastructure"]
org: Julius-Maximilians-Universität Würzburg
fetched_at: 2026-05-16T02:57:45.660Z
classified_at: 2026-05-16T02:57:45.624Z
abstract:
We study inventory control with decision-dependent censoring, focusing on the censored or repeated newsvendor (R-NV), where each order quantity determines whether demand is fully observed or censored by sales. Existing approaches based on parametric Thompson sampling (TS) can be brittle under prior mismatch, while offline imputation methods need not transfer to online learning. Motivated by the predictive view of decision making, we combine these ideas by taking oracle actions on learned completions of latent demand. We propose in-context generative posterior sampling (ICGPS), which uses modern generative models that are meta-trained offline and deployed online by in-context autoregressive generation. Theoretically, we show that the Bayesian regret of ICGPS with a learned completion kernel is bounded by the Bayesian regret of a TS benchmark with the ideal completion kernel plus a deployment penalty scaling as $\sqrt{T}$ times the square root of the completion mismatch. This yields a plug-in template for operational problems with known TS regret bounds. For R-NV, we derive sublinear Bayesian regret by reducing censored feedback to bandit convex optimization feedback. We also show that, under reasonable coverage and stability assumptions, the online completion mismatch is controlled by the offline censored predictive mismatch, so offline predictive quality transfers to online performance. Practically, we instantiate ICGPS with ChronosFlow, which combines a frozen time-series transformer backbone with a trainable conditional normalizing-flow head for fast censoring-consistent sampling. In benchmark experiments, ChronosFlow-ICGPS matches correctly specified TS, outperforms myopic and UCB-style baselines, and is robust to prior mismatch and distribution shift. ChronosFlow-ICGPS also performs well for the real-world SuperStore dataset, especially under heavy censoring.


source_id: 2605.14828
source: arxiv
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14828v1
tags: ["theories and proofs","evaluation","interpretability"]
org: Politecnico di Milano
fetched_at: 2026-05-16T02:59:00.861Z
classified_at: 2026-05-16T02:59:00.831Z
title:
K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles

authors:
["Giulia Patanè","Alessandra Menafoglio","Alexander Krauth","Peter Fechner","Luca Dede'","Bianca Maria Colosimo","Federica Nicolussi"]

abstract:
Existing clustering methods for functional data often prioritize partitioning accuracy over interpretability, making it challenging to extract meaningful insights when the data-generating process follows a specific underlying structure and an ordinal relationship among clusters is suspected. This work introduces K-Models, a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles, improving both interpretability and structure identification. The proposed method is evaluated through simulations and real-world applications. In particular, it is tested on Region of Interest (ROI) curves, which represent reaction profiles from a reflectometric sensor monitoring biomolecular interactions, such as antigen-antibody binding. These curves represent changes in reflected light intensity over time at multiple measurement spots with immobilized antigens during analyte exposure, capturing the binding dynamics of the system. The goal is to identify intrinsic signal patterns solely from the observed dynamics, making this dataset an ideal benchmark for assessing the added interpretability of the proposed approach. By incorporating structural assumptions into the clustering process, K-Models enhances interpretability while maintaining performance comparable to state-of-the-art techniques, providing a valuable tool for analyzing functional data with an underlying ordinal structure.


source_id: 2605.14567
source: arxiv
title: Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
authors: ["Arie Wortsman-Zurich","Hugo Tabanelli","Yatin Dandi","Florent Krzakala","Bruno Loureiro"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14567v1
tags: ["theories and proofs","training","evaluation"]
org: Stanford University
fetched_at: 2026-05-16T03:00:33.756Z
classified_at: 2026-05-16T03:00:33.721Z
abstract:
We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-degree function, but that can be represented by a combination of latent compositional features whose weights decrease as a power law. We show that a layer-wise spectral algorithm adapted to this compositional structure achieves improved scaling relative to shallow, non-adaptive methods, and recovers the latent directions sequentially: strong features become detectable at small sample sizes, while weaker features require more data. We prove sharp feature-wise recovery thresholds and show that aggregating these transitions yields an explicit power-law decay of the prediction error. Technically, the analysis relies on random matrix methods and a resolvent-based perturbation argument, which gives matching upper and lower bounds for individual eigenvector recovery beyond what standard gap-based perturbation bounds provide. Numerical experiments confirm the predicted sequential recovery, finite-size smoothing of the thresholds, and separation from non-hierarchical kernel baselines. Together, these results show how smooth scaling laws can emerge from a cascade of sharp feature-learning transitions.


source_id: 2605.14524
source: arxiv
title: Large Dimensional Kernel Ridge Regression: Extending to Product Kernels
authors: ["Yang Zhou","Yicheng Li","Yuqian Cheng","Qian Lin"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14524v1
tags: ["theories and proofs","information geometry","evaluation"]
org: Tsinghua University
fetched_at: 2026-05-16T03:02:00.661Z
classified_at: 2026-05-16T03:02:00.628Z
abstract:
Recent studies have reported $\textit{saturation effects}$ and $\textit{multiple descent behavior}$ in large dimensional kernel ridge regression (KRR). However, these findings are predominantly derived under restrictive settings, such as inner product kernels on sphere or strong eigenfunction assumptions like hypercontractivity. Whether such behaviors hold for other kernels remains an open question. In this paper, we establish a broad, new family of large dimensional kernels and derive the corresponding convergence rates of the generalization error. As a result, we recover key phenomena previously associated with inner product kernels on sphere, including: $i)$ the $\textit{minimax optimality}$ when the source condition $s\le 1$; $ii)$ the $\textit{saturation effect}$ when $s>1$; $iii)$ a $\textit{periodic plateau phenomenon}$ in the convergence rate and a $\textit {multiple-descent behavior}$ with respect to the sample size $n$.


source_id: 2605.14599
source: arxiv
title: Fast Rates for Inverse Reinforcement Learning
authors: ["Andreas Schlaginhaufen","Maryam Kamgarpour"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14599v1
tags: ["theories and proofs","reinforcement learning","evaluation"]
org: Sycamore, EPFL
fetched_at: 2026-05-16T03:03:24.008Z
classified_at: 2026-05-16T03:03:23.976Z
abstract:
We establish novel structural and statistical results for entropy-regularized min-max inverse reinforcement learning (Min-Max-IRL) with linear reward classes in finite-horizon MDPs with Borel state and action spaces. On the structural side, we show that maximum likelihood estimation (MLE) and Min-Max-IRL are equivalent at the population level, and at the empirical level under deterministic dynamics. On the statistical side, exploiting pseudo-self-concordance of the Min-Max-IRL loss, we prove that both the trajectory-level KL divergence and the squared parameter error in the Hessian norm decay at the fast rate $\mathcal{O}(n^{-1})$, where $n$ is the number of expert trajectories. Our guarantees apply under misspecification and require no exploration assumptions. We further extend reward-identifiability results to general Borel spaces and derive novel results on the derivatives of the soft-optimal value function with respect to reward parameters.


source_id: 2605.14343
source: arxiv
title: Nearest-Neighbor Radii under Dependent Sampling
authors: ["Yuanyuan Gao","Yilong Hou","Zhexiao Lin"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14343v1
tags: ["theories and proofs","information geometry","evaluation"]
org: University of California, Berkeley
fetched_at: 2026-05-16T03:04:34.354Z
classified_at: 2026-05-16T03:04:34.320Z
abstract:
Nearest-neighbor methods are fundamental to classical and modern machine learning, yet their geometric properties are typically analyzed under independent sampling. In this paper, we study the nearest-neighbor radii under dependent sampling. We consider strong mixing dependent observations and ask whether dependence changes the scale of nearest-neighbor neighborhoods. We establish distribution-free almost sure convergence under polynomial mixing and sharp non-asymptotic moment bounds under geometric mixing. The moment bounds depend on the local intrinsic dimension rather than the ambient dimension, making the results applicable to high-dimensional data concentrated near lower-dimensional manifolds. Synthetic experiments and real-world time-series benchmarks support the theory, showing that nearest-neighbor geometry remains informative under dependence sampling.


source_id: 2605.14297
source: arxiv
title: Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradients
authors: ["Matias Alvo","Daniel Russo","Yash Kanoria"]
published_date: 2026-05-14
pdf_url: https://arxiv.org/pdf/2605.14297v1
tags: ["reinforcement learning","theories and proofs","evaluation"]
org: Stanford University
fetched_at: 2026-05-16T03:05:54.813Z
classified_at: 2026-05-16T03:05:54.783Z
abstract:
We study reinforcement learning in hybrid discrete-continuous action spaces, such as settings where the discrete component selects a regime (or index) and the continuous component optimizes within it -- a structure common in robotics, control, and operations problems. Standard model-free policy gradient methods rely on score-function (SF) estimators and suffer from severe credit-assignment issues in high-dimensional settings, leading to poor gradient quality. On the other hand, differentiable simulation largely sidesteps these issues by backpropagating through a simulator, but the presence of discrete actions or non-smooth dynamics yields biased or uninformative gradients. To address this, we propose Hybrid Policy Optimization (HPO), which backpropagates through the simulator wherever smoothness permits, using a mixed gradient estimator that combines pathwise and SF gradients while maintaining unbiasedness. We also show how problems with action discontinuities can be reformulated in hybrid form, further broadening its applicability. Empirically, HPO substantially outperforms PPO on inventory control and switched linear-quadratic regulator problems, with performance gaps increasing as the continuous action dimension grows. Finally, we characterize the structure of the mixed gradient, showing that its cross term -- which captures how continuous actions influence future discrete decisions -- becomes negligible near a discrete best response, thereby enabling approximate decentralized updates of the continuous and discrete components and reducing variance near optimality. All resources are available at github.com/MatiasAlvo/hybrid-rl.


