TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention
Follow the cross-layer propagation of harmful semantics and intervene at the pathway level.
The short version
TraceRouter studies safety through paths connecting internal components. It localizes sensitive onset layers, uses sparse features to trace relevant attention interactions, and applies path-level suppression through a focused intervention. The goal is to interrupt the information flow associated with unsafe behavior while preserving general model utility.
The work evaluates both diffusion and language models, pairing safety measurements with quality or utility measurements. Its analysis shifts attention from where a feature exists to how that feature participates in downstream computation.
A behavior can depend on a route through the network, making information flow a useful unit of intervention.
Locating the sensitive onset
Layer-wise sensitivity scores identify candidate onset layers across three model families.
View in the paperWhy this matters
Distributed harmful computation can continue through multiple layers even when a local feature is suppressed. A defence therefore needs to examine how the information propagates.
How it works
- Locate a sensitive onset layer through attention-divergence analysis.
- Use SAEs and differential activations to isolate malicious features.
- Estimate feature influence with zero-out interventions and suppress selected downstream pathways.
Safety across model families
Original model-level comparisons report safety outcomes on multimodal language models.
View in the paperWhat the study finds
Reported comparisons examine adversarial robustness together with general utility. The results support the value of tracing and suppressing selected propagation pathways rather than treating safety as a purely local feature intervention.
Safety under several attack settings
The table compares interventions across gradient-based and pattern-based attacks on language models.
View in the paperWhat this means
This gives interpretability a more relational target. Two components can matter because of the route linking them, even when neither isolated component fully explains the behavior. Tracing that route can make an intervention more specific about where and how it acts.
The practical challenge is selectivity across tasks and model families. Path-level evidence offers a way to reason about that selectivity, while the reported evaluations define the settings in which suppressing the selected routes improves safety with limited utility loss.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@misc{tracerouter,
title = {TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention},
author = {Shi, Chuancheng and Li, Shangze and Lu, Wenjun and Wu, Wenhua and Wang, Cong and Cheng, Zifeng and Shen, Fei and Chua, Tat-Seng},
year = {2026},
eprint = {2601.21900},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2601.21900}
}