NExT-LFM · Research overview
More research
Fundamental Research / Safety pathways / Research overview

TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport

Trace safety-relevant causal routes and softly realign selected activation distributions toward safe references using a coordinate-wise optimal transport approximation.

Shi Chuancheng

TPAMI 2026CCF-A 4-minute read Read the paper PDF
Concept illustration: TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport
Concept illustration inspired by the study.

The short version

An unsafe behavior can travel through a network rather than reside in one isolated neuron. TraceRouter++ identifies safety-relevant routes and changes how their activations are distributed. The aim is to steer those routes toward safe-reference behavior while limiting the disturbance introduced at the intervention point.

The journal extension adds optimal-transport-based soft realignment to the Discover–Trace–Realign pipeline. It uses tractable coordinate-wise quantile mappings: this aligns selected marginals, rather than solving a general joint transport problem.

99.5%Mean LLaMA3 defense success rate
3 model familiesLLMs, MLLMs, and diffusion models
Optimal transportSoft activation realignment

Identifying a harmful route and deciding how to change its activation are complementary parts of safety intervention.

Figure 2 · Discover, trace, and realign. The original framework links onset localization, sparse routing features, causal influence tracing, and optimal transport realignment.
Figure 2 / p. 3

Discover, trace, and realign

The original framework links onset localization, sparse routing features, causal influence tracing, and optimal transport realignment.

View in the paper

Why this matters

Suppressing an isolated feature can leave alternative propagation routes available and can also disturb useful computation. A route-level intervention makes both the location and the form of the modification explicit: where safety-relevant information travels, and how the selected activations should change.

How it works

  1. Discover a sensitive onset layer from attention divergence, then use a sparse autoencoder to isolate interpretable routing features.
  2. Trace downstream causal influence with feature influence scores, selecting routes supported by intervention evidence.
  3. Realign the selected activation marginals toward safe-reference marginals with an optimal transport approximation, leaving unselected coordinates unchanged at that intervention point.
Table 2 · Defense against several language-model jailbreaks. The table reports defense success rates, including 99.5 ± 0.3% and 98.8 ± 0.3% averages on the two evaluated LLMs.
Table 2 / p. 8

Defense against several language-model jailbreaks

The table reports defense success rates, including 99.5 ± 0.3% and 98.8 ± 0.3% averages on the two evaluated LLMs.

View in the paper

What the study finds

On the reported LLM attack suite, average defense success rates reach 99.5 ± 0.3% for LLaMA3-8B-Instruct and 98.8 ± 0.3% for Mistral-7B-Instruct. The paper also evaluates multimodal language models and diffusion models.

Utility measurements accompany the safety results: Global-MMLU-Lite changes by −0.4 and −0.6 percentage points on the two language models. These empirical trade-offs are distinct from the local guarantee that unselected activation coordinates are preserved.

Figure 8 · Comparing path-level and neuron-level interventions. The ablation compares routing realignment with isolated neuron suppression across language, multimodal, and diffusion settings.
Figure 8 / p. 12

Comparing path-level and neuron-level interventions

The ablation compares routing realignment with isolated neuron suppression across language, multimodal, and diffusion settings.

View in the paper

What this means

The insight is to treat safety control as a problem of shaping information flow. Causal tracing identifies a target for intervention, while distributional realignment provides an alternative to simply deleting the target signal. This connects mechanistic analysis with a more selective control objective.

The boundary is equally informative. Coordinate-wise transport does not generally preserve joint dependencies, and later layers can mix selected and unselected coordinates. Safety and utility therefore remain outcomes to measure, rather than consequences guaranteed by local sparsity alone.

Paper and resources

Contact the authorShi Chuancheng
cshi0459@uni.sydney.edu.au

This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.

Cite this work
@article{tracerouter_plus_plus,
  title = {TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport},
  author = {Shi, Chuancheng and Li, Shangze and Wu, Wenhua and Chen, Hengzhi and Shen, Fei and Zhuo, Li and Wang, Zhiyong and Chua, Tat-Seng},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  doi = {10.1109/TPAMI.2026.3747849},
  url = {https://doi.org/10.1109/TPAMI.2026.3747849}
}