Modulate-and-Map: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection

Alex Costanzino1, Pierluigi Zama Ramirez2, Giuseppe Lisanti1, Luigi Di Stefano1,
1CVLab, University of Bologna, 2Ca' Foscari University of Venice
Accepted at CVPR Findings 2026
Paper arXiv Code Hugging Face Data
Teaser image

We extend Crossmodal Feature Mapping with Feature-wise Linear Modulations to enable cross-view training.

Abstract

We present ModMap, a natively multiview and multimodal framework for 3D anomaly detection and segmentation. Unlike existing methods that process views independently, our method draws inspiration from the crossmodal feature mapping paradigm to learn to map features across both modalities and views, while explicitly modelling view-dependent relationships through feature-wise modulation.

We introduce a cross-view training strategy that leverages all possible view combinations, enabling effective anomaly scoring through multiview ensembling and aggregation. To process high-resolution 3D data, we train and publicly release a foundational depth encoder tailored to industrial datasets

Experiments on SiM3D, a recent benchmark that introduces the first multiview and multimodal setup for 3D anomaly detection and segmentation, demonstrate that ModMap attains state-of-the-art performance by surpassing previous methods by wide margins.

BibTeX

@article{costanzino2026modmap,
  author    = {Costanzino, Alex and Zama Ramirez, Pierluigi and Lisanti, Giuseppe and Di Stefano, Luigi},
  title     = {Modulate-and-Map: Crossmodal Feature Mapping with Cross-View Modulation for 3D Anomaly Detection},
  journal   = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition Findings},
  year      = {2026},
}