Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities
MAF learns a shared MLP detector across modalities and probes neuron co-activation to examine shared latent forensic knowledge.
The short version
Beyond Surface Artifacts studies forgery detection across modalities rather than only across generators within one medium. Its MAF framework uses modality-specific perceptors and a shared lightweight detector, with regularization intended to reduce dependence on modality-specific style and encourage forensic alignment.
Two evaluation settings separate different transfer conditions. Weak MAF uses semantically aligned perceptors; Strong MAF uses an independently learned perceptor for an unseen modality. In the latter setting, target-modality data can support self-supervised representation learning while the forgery detector itself remains fixed.
Forensic transfer asks what remains informative when the physical medium and its familiar artifacts change.
Separating detector reuse from the perceptor
Weak and Strong MAF expose different assumptions about the representation of an unseen modality.
View in the paperWhy this matters
A representation aligned for semantics need not align the distinctions relevant to forensics. A detector can therefore appear broadly capable while depending on artifacts tied to the media it has already seen.
How it works
- Encode source modalities with their perceptors and train a shared lightweight forgery detector.
- Combine classification with cross-modal regularization to encourage transferable forensic representations.
- Evaluate unseen modalities under Weak and Strong MAF, explicitly distinguishing the target perceptor and data assumptions.
Semantic alignment and forensic alignment
The original visualizations compare modality structure in semantic and forensic feature spaces.
View in the paperWhat the study finds
DeepModal-Bench compares cross-modality generalization across video, audio, and image settings. The selected figures contrast semantic and forensic feature distributions and report Strong MAF performance across several pretrained perceptors. Further analyses inspect distribution consistency and cross-modal neuron co-activation.
Testing the stronger transfer setting
The bars compare detection accuracy in Strong MAF across several perceptors and benchmarks.
View in the paperWhat this means
This work broadens the unit of generalization from a new generator to a new kind of signal. It invites a more demanding question about detection: which learned distinctions reflect the task, and which merely reflect the medium through which the task was observed?
The two settings make the assumptions of that question visible. Performance with an aligned perceptor and performance with an independently learned target representation support different conclusions. Keeping those conditions explicit is essential when assessing how much forensic knowledge genuinely transfers.
Paper and resources
This is an introduction to the linked paper. Refer to the original for method definitions, evaluation settings, and complete results.
Cite this work
@misc{beyond_surface_artifacts,
title = {Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities},
author = {Dou, Jingtong and Shi, Chuancheng and Wang, Jian and Shen, Fei and Wang, Zhiyong and Chua, Tat-Seng},
year = {2026},
eprint = {2604.07763},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2604.07763}
}