Journal
JOURNAL OF LIGHTWAVE TECHNOLOGY
Volume 37, Issue 7, Pages 1742-1749Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JLT.2019.2902487
Keywords
Data clustering module (DCM); data regression and classification module (DRCM); hybrid unsupervised and supervised machine learning; self-taught anomaly detection
Funding
- Department of Energy [DE-SC0016700]
- National Science Foundation ICE-T: RC [1836921]
- U.S. Department of Energy (DOE) [DE-SC0016700] Funding Source: U.S. Department of Energy (DOE)
- Direct For Computer & Info Scie & Enginr [1836921] Funding Source: National Science Foundation
- Division Of Computer and Network Systems [1836921] Funding Source: National Science Foundation
Ask authors/readers for more resources
This paper proposes a self-taught anomaly detection framework for optical networks. The proposed framework makes use of a hybrid unsupervised and supervised machine learning scheme. First, it employs an unsupervised data clustering module (DCM) to analyze the patterns of monitoring data. The DCM enables a self-learning capability that eliminates the requirement of prior knowledge of abnormal network behaviors and therefore can potentially detect unforeseen anomalies. Second, we introduce a self-taught mechanism that transfers the patterns learned by the DCM to a supervised data regression and classification module (DRCM). The DRCM, whose complexity is mainly related to the scale of the applied supervised learning model, can potentially facilitate more scalable and time-efficient online anomaly detection by avoiding excessively traversing the original dataset. We designed the DCM and DRCM based on the density-based clustering algorithm and the deep neural network structure, respectively. Evaluations with experimental data from two use cases (i.e., single-point detection and end-to-end detection) demonstrate that up to 99% anomaly detection accuracy can be achieved with a false positive rate below 1%.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available