Revisiting Over-refusal in Safety-aligned Large Language Models: A Representation and Optimization Perspective
GOAL combines gradient orthogonalization and contrastive representation learning to improve the boundary between harmful and benign risky-looking requests.
The short version
This work asks why harmless requests can still be refused after safety training with correct supervision. It examines two interacting observations: harmful and over-refusal representations become less distinguishable at deeper layers, while harmful-sample gradients concentrate in a low-rank subspace and align more strongly with refusal-target updates.
GOAL responds at both levels. It projects over-refusal gradients away from the harmful-gradient refusal subspace and uses a contrastive objective to strengthen representational separation. The method connects a diagnosis of training dynamics to a targeted change in how the model learns the safety boundary.
Accurate safety labels need an optimization process that preserves the distinction they are meant to teach.
Two interventions in the learning process
GOAL combines refusal-subspace gradient projection with safety-aware contrastive learning.
View in the paperWhy this matters
Correctly labeling a benign request does not ensure that its desired update will be preserved during joint training. Understanding over-refusal requires examining how examples interact through both their representations and the gradients that change the model.
How it works
- Estimate a low-rank refusal subspace from harmful-sample gradients.
- Orthogonalize over-refusal gradients relative to that subspace to reduce conflicting update components.
- Combine the gradient intervention with contrastive learning that separates harmful and over-refusal representations.
Examining gradient coupling
The original analyses compare low-rank concentration and gradient alignment across sample groups.
View in the paperWhat the study finds
Representation visualizations and gradient analyses motivate the two components. Experiments across three base models compare GOAL with alignment and over-refusal baselines using joint measures of over-refusal, harmful-request safety, and general capabilities. Component ablations and subspace-sensitivity analyses test the proposed design.
Calibration requires joint evaluation
The table compares over-refusal, harmful-request safety, and general-capability measurements across three models.
View in the paperWhat this means
The study treats calibration as a property of the learning process, alongside the data supplied to it. Even well-supervised examples can interact in ways that erode a useful boundary, so improving safety may require controlling the geometry of those interactions.
This perspective links behavior to two testable targets: what distinctions remain visible internally, and which directions training updates reinforce. Evaluating both helps explain why a method changes refusal patterns, while joint safety and utility measurements establish whether the resulting calibration is useful.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@misc{revisiting_over_refusal,
title = {Revisiting Over-refusal in Safety-aligned Large Language Models: A Representation and Optimization Perspective},
author = {Chen, Chao and Shao, Pengyang and Qin, Wei and Hao, Shijie and Shen, Fei and Hong, Richang},
year = {2026},
eprint = {2610.39802},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.39802}
}