How L1* Regularization Transforms Multi-Modal Anomaly Detection: A New Standard for Semantic Accuracy

Global, Source:

In a world where data streams from countless sources, traditional anomaly detection often struggles to keep pace. Most existing algorithms rely on single-modal inputs like text or images alone, missing the intricate correlations that define real-world scenarios. This limitation leads to inaccurate decisions when anomalies are subtle or masked within diverse data streams. To address this critical gap, researchers have introduced MSENet, a novel end-to-end multi-modal semantic-enhanced network designed specifically for robust novelty detection.

MSENet operates on the principle that normal data shares common semantic information across different modalities, while anomalies disrupt this harmony. The architecture consists of two primary modules: a feature-extraction network and a semantic-mining network. During training, the model processes multi-modal inputs such as text, video, and audio through asymmetric autoencoders tailored to each modality's specific dimensions. These extracted features are then fused using a concatenation operation and passed to the semantic-mining module. Here, the network learns to extract shared semantic representations, effectively bridging the semantic gaps between disparate data types. These mined semantics are subsequently fused back with the original features to create semantic-enhanced representations, which are then reconstructed. The core assumption is that the model will reconstruct normal samples with high fidelity but struggle to accurately reconstruct anomalous data, thereby flagging them based on reconstruction error.

A key innovation in this framework is the introduction of L1* regularization. While standard L1 regularization promotes sparsity by driving weights to zero, it can sometimes be too aggressive or rigid. L1* introduces a flexible penalty term based on the L1-norm, reshaped to encourage sparser weights during optimization without overly constraining the model. This method selects more representative features, ensuring that the network focuses on the most informative aspects of the data rather than noise or irrelevant variations. By combining MSENet’s semantic enhancement with L1*’s feature selection, the system achieves superior performance in identifying novelties.

Experimental validation was conducted on two prominent multi-modal datasets: MUStARD (Multi-modal Sarcasm Detection) and UR-FUNNY (Universal-Funny). In these tests, sarcasm and humor were treated as the 'normal' classes, while non-sarcastic and non-humorous content served as anomalies. The results demonstrated that MSENet with L1* regularization outperformed state-of-the-art methods, including Deep Autoencoders (DAE), One-Class Support Vector Machines (OC-SVM), Kernel Density Estimation (KDE), and Deep SVDD. On the MUStARD dataset, the proposed method achieved an Area Under the Curve (AUC) score of 69.956%, significantly higher than the next best competitor. Ablation studies confirmed that both the semantic-mining module and the L1* regularizer contribute independently to the model’s effectiveness, with statistical analysis verifying the significance of these improvements.

Despite strong performance on MUStARD, the method faced challenges on the UR-FUNNY dataset, where all models performed close to chance levels. Analysis revealed that this was due to the dataset’s inherent structure; negative samples were drawn from the same videos as positive ones, sharing identical speakers and recording conditions. Without contextual cues like laughter markers, the visual and audio signals lacked sufficient discriminative power, highlighting the importance of context in multi-modal learning. Nevertheless, the success on MUStARD underscores the potential of semantic-enhanced architectures in handling complex, real-world data. This approach offers a universal framework applicable to various fields, from industrial quality control to meteorological satellite monitoring, paving the way for more reliable and intelligent anomaly detection systems.