Feedback-Aware Anomaly Detection Through Logs for Large-Scale Software Systems

Release Date:2021-09-16 Author:HAN Jing, JIA Tong, WU Yifan, HOU Chuanjia, LI Ying Click:

 

Feedback-Aware Anomaly Detection Through Logs for Large-Scale Software Systems
 
HAN Jing1, JIA Tong2, WU Yifan2,HOU Chuanjia2,LI Ying2
(1. ZTE Corporation, Shenzhen 518057, China;
2. Peking University, Beijing 100091, China)
 

Abstract: 

One particular challenge for large-scale software systems is anomaly detection. System logs are a straightforward and common source of information for anomaly detection. Existing log-based anomaly detectors are unusable in real-world industrial systems due to high false-positive rates. In this paper, we incorporate human feedback to adjust the detection model structure to reduce false positives. We apply our approach to two industrial large-scale systems. Results have shown that our approach performs much better than state-of-the-art works with 50% higher accuracy. Besides, human feedback can reduce more than 70% of false positives and greatly improve detection precision.


Keywords: human feedback; log-based anomaly detection; system log

 

Download: PDF