TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport
Trace safety-relevant causal routes and softly realign selected activation distributions toward safe references using a coordinate-wise optimal transport approximation.
The short version
An unsafe behavior can travel through a network rather than reside in one isolated neuron. TraceRouter++ identifies safety-relevant routes and changes how their activations are distributed. The aim is to steer those routes toward safe-reference behavior while limiting the disturbance introduced at the intervention point.
The journal extension adds optimal-transport-based soft realignment to the Discover–Trace–Realign pipeline. It uses tractable coordinate-wise quantile mappings: this aligns selected marginals, rather than solving a general joint transport problem.
Identifying a harmful route and deciding how to change its activation are complementary parts of safety intervention.
Discover, trace, and realign
The original framework links onset localization, sparse routing features, causal influence tracing, and optimal transport realignment.
View in the paperWhy this matters
Suppressing an isolated feature can leave alternative propagation routes available and can also disturb useful computation. A route-level intervention makes both the location and the form of the modification explicit: where safety-relevant information travels, and how the selected activations should change.
How it works
- Discover a sensitive onset layer from attention divergence, then use a sparse autoencoder to isolate interpretable routing features.
- Trace downstream causal influence with feature influence scores, selecting routes supported by intervention evidence.
- Realign the selected activation marginals toward safe-reference marginals with an optimal transport approximation, leaving unselected coordinates unchanged at that intervention point.
Defense against several language-model jailbreaks
The table reports defense success rates, including 99.5 ± 0.3% and 98.8 ± 0.3% averages on the two evaluated LLMs.
View in the paperWhat the study finds
On the reported LLM attack suite, average defense success rates reach 99.5 ± 0.3% for LLaMA3-8B-Instruct and 98.8 ± 0.3% for Mistral-7B-Instruct. The paper also evaluates multimodal language models and diffusion models.
Utility measurements accompany the safety results: Global-MMLU-Lite changes by −0.4 and −0.6 percentage points on the two language models. These empirical trade-offs are distinct from the local guarantee that unselected activation coordinates are preserved.
Comparing path-level and neuron-level interventions
The ablation compares routing realignment with isolated neuron suppression across language, multimodal, and diffusion settings.
View in the paperWhat this means
The insight is to treat safety control as a problem of shaping information flow. Causal tracing identifies a target for intervention, while distributional realignment provides an alternative to simply deleting the target signal. This connects mechanistic analysis with a more selective control objective.
The boundary is equally informative. Coordinate-wise transport does not generally preserve joint dependencies, and later layers can mix selected and unselected coordinates. Safety and utility therefore remain outcomes to measure, rather than consequences guaranteed by local sparsity alone.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@article{tracerouter_plus_plus,
title = {TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport},
author = {Shi, Chuancheng and Li, Shangze and Wu, Wenhua and Chen, Hengzhi and Shen, Fei and Zhuo, Li and Wang, Zhiyong and Chua, Tat-Seng},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
doi = {10.1109/TPAMI.2026.3747849},
url = {https://doi.org/10.1109/TPAMI.2026.3747849}
}