A Search-based Approach for Accurate Identification of Log Message Formats

Salma Messaoudi, Annibale Panichella, Domenico Bianculli, Lionel Briand, Raimondas Sasnauskas

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

82 Citations (Scopus)
71 Downloads (Pure)

Abstract

Many software engineering activities process the events contained in log files. However, before performing any processing activity, it is necessary to parse the entries in a log file, to retrieve the actual events recorded in the log. Each event is denoted by a log message, which is composed of a fixed part—called (event) template—that is the same for all occurrences of the same event type, and a variable part, which may vary with each event occurrence. The formats of log messages, in complex and evolving systems, have numerous variations, are typically not entirely known, and change on a frequent basis; therefore, they need to be identified automatically. The log message format identification problem deals with the identification of the different templates used in the messages of a log. Any solution to this problem has to generate templates that meet two main goals: generating templates that are not too general, so as to distinguish different events, but also not too specific, so as not to consider different occurrences of the same event as following different templates; however, these goals are conflicting. In this paper, we present the approach approach, which recasts the log message identification problem as a multi-objective problem. approach uses an evolutionary approach to solve this problem, by tailoring the NSGA-II algorithm to search the space of solutions for a Pareto optimal set of message templates. We have implemented approach in a tool, which we have evaluated on six real-world datasets, containing log files with a number of entries ranging from 2K to 300K. The experiments results show that approach extracts by far the highest number of correct log message templates, significantly outperforming two state-of-the-art approaches on all datasets.
Original languageEnglish
Title of host publicationProceedings of the 26th International Conference on Program Comprehension
Place of PublicationNew York, NY
PublisherACM/IEEE
Pages167-177
Number of pages11
ISBN (Electronic)978-1-4503-5714-2
DOIs
Publication statusPublished - 2018
EventICPC 2018: IEEE/ACM International Conference on Program Comprehension 2018 - Gothenburg, Sweden
Duration: 27 May 201828 May 2018
https://conf.researchr.org/home/icpc-2018

Conference

ConferenceICPC 2018
Country/TerritorySweden
CityGothenburg
Period27/05/1828/05/18
OtherCo-located with ICSE 2018
Internet address

Bibliographical note

Accepted Author Manuscript

Keywords

  • log parsing
  • log analysis
  • log message
  • format
  • NSGA-II

Fingerprint

Dive into the research topics of 'A Search-based Approach for Accurate Identification of Log Message Formats'. Together they form a unique fingerprint.

Cite this