Loading
Life sciences · Preprint
arXiv · August 19, 2026
Posted before peer review. The findings may change or fail to hold.
This preprint presents a novel Interaction-based Prompt Sensitivity (IPS) metric that decomposes LLM outputs into interaction terms to reveal fine-grained instability in model behaviour. Empirical validation across 50 open-source models identifies four factors—supervised fine-tuning, model scale, dense architecture, and few-shot learning—that reduce prompt sensitivity by stabilizing low-order interactions. The work is methodologically innovative but lacks peer review and does not compare against existing prompt sensitivity metrics.
Methodological framework with empirical validation across multiple models. 50 open-source LLMs. Intervention: Interaction-based Prompt Sensitivity (IPS) metric quantifying changes in interactions when prompts are subtly modified.
Subtle, semantically irrelevant prompt changes trigger severe instability in interactions even when LLM outputs remain unchanged. Four factors reduce prompt sensitivity: supervised fine-tuning, increased model scale, dense architectures, and few-shot learning. All four factors operate via a common mechanism: reduction of low-order interaction instability.
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is an unrefereed arXiv preprint introducing a novel analytical framework (IPS metric) for understanding LLM prompt sensitivity through interaction decomposition, with empirical validation across 50 models but no peer review.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
The remarkable capabilities of large language models (LLMs) are often undermined by their instability. Even subtle and semantically irrelevant changes in prompts can cause dramatic fluctuations in performance, a phenomenon known as prompt sensitivity. Previous studies typically evaluate prompt sensitivity by comparing the LLM's final outputs when prompts change. However, such coarse-grained metrics fail to explain the internal reasons for prompt sensitivity. In this paper, we introduce interactions as a fine-grained tool to analyze prompt sensitivity of LLMs. Specifically, we decompose the output score of the LLM into a set of interactions. Each interaction represents a nonlinear relationship involving a set of input variables. We discover that subtle changes to prompts can trigger severe instability in interactions, even when the outputs of the LLM remain the same. To this end, we propose an Interaction-based Prompt Sensitivity (IPS) metric by quantifying changes in interactions when we introduce subtle changes to prompts. We apply the IPS metric to 50 open-source LLMs and uncover four factors that reduce the prompt sensitivity of LLMs, including supervised fine-tuning, increased model scales, dense architectures, and few-shot learning. More crucially, we discover a common mechanism by which these four factors reduce prompt sensitivity: all four factors tend to reduce the prompt sensitivity of low-order interactions (i.e., interactions involving few input variables).
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.