Monday, August 15, 2011

Log Files Filter Utility

Overview:

This open source project aim to filter the log files according to certain criteria and produce small piece of log file according to the given criteria , this is very useful in case of huge log files.

The output is another log file according to your needs (time range + search /exclude keywords).

Features:

Here is some of the supported features:

* Parse and filter huge log files using time range.
* Configurable log file date/time format.
* Filter by keywords or exclude lines with certain keywords.
* Can filter errors only in this time range if needed.
* Optionally zip the output file.
* The only mandatory parameter is the log file name, if date is missing today date will be used, start time default to 00:00:00 , end time default to 23:59:59 (both are configurable)


Download:

http://sourceforge.net/projects/logfilterutil/


Usage:

-Ffilenanme : specify the log file name, the only mandatory parameter
-Ddate : specify the date in the log file
-Ttime : specify the start time and end time in the log file (1st T for start time and 2nd for end time)
-Ooutput file : specify the output log file name
-Skey : to search for this key in the log and construct the output file
-Xkey : to exclude the lines contain this key word from the log and construct the output file
-E : if you want the Errors only, this will have no effect if search/exclude parameter is used.
-Z : if you want to zip the output file
You can use either -E or -S or -X and -S or -X has the superiority over -E

The default log file date/time-stamp is "yyyy-MM-dd HH:mm:ss" , you can configure it according to the log file used format.

**Example :

java -jar LogFilterUtility.jar -Z -E -Sstuck -D2011-08-10 -T04:00:00 -T06:00:00 -Flogfile.log

Future Work:

- Split log file according to specified max size of each split part.
- Export HTML file to color the log file according to the log level.

3 comments:

  1. Hey Osama,

    This is a simple yet effective tool - Thanks for it .
    While running the tool i faced issue and everytime the it throws "Error: Parsing error! will skip this line."

    I did went on and looked at the class file and looks like the code is not able to parse the date instead it is considering the whole line as a string.

    Any help is much appreciated!.

    String input = in.readLine();
    long currentDate = 0L;
    while (input != null)
    {
    try
    {
    currentDate = sdf.parse(input).getTime();
    }
    catch (Throwable t)
    {

    System.out.println("Error: Parsing error! will skip this line.");
    currentDate = 0L;
    }

    ReplyDelete
    Replies
    1. The default log file date/time-stamp is "yyyy-MM-dd HH:mm:ss" , you can configure it according to the log file used format otherwise you will get this parsing error.

      Delete
    2. There is a configuration file: logTimeFormat.txt you need to change it for that purpose.

      Delete