Event Normalization Through Dynamic Log Format Detection

Release Date:2014-10-08 Author:Amir Azodi, David Jaeger, Feng Cheng, and Christoph Meinel Click:

[Abstract] The analytical and monitoring capabilities of central event repositories, such as log servers and intrusion detection systems, are limited by the amount of structured information extracted from the events they receive. Diverse networks and applications log their events in many different formats, and this makes it difficult to identify the type of logs being received by the central repository. The way events are logged by IT systems is problematic for developers of host⁃based intrusion⁃detection systems (specifically, host⁃based systems), developers of security⁃information systems, and developers of event⁃management systems. These problems preclude the development of more accurate, intrusive security solutions that obtain results from data included in the logs being processed. We propose a new method for dynamically normalizing events into a unified super⁃event that is loosely based on the Common Event Expression standard developed by Mitre Corporation. We explain how our solution can normalize seemingly unrelated events into a single, unified format.

[Keywords] event normalization; intrusion detection; event stream processing; knowledge base; security information and event management