Tools used for parsing large log files


   In this article I will talk about the tools that I use when parsing large log files. A couple of days ago I had a problem with a service that was logging around 3 GB data per day. With small sized log files there is no problem processing them because this can be done easily using notepad or notepad++. From what I know, these tools are limited to files that are less than 2GB. There are other utilities available from third party vendors that can parse log files of all sizes. I use several tools for processing log files but, among the best are the following:
   Baretail – freeware parsing tool available for download form here. Baretail is easy to use and has a friendly graphical interface:
Baretail tool
I like this tool especially because you can use it to follow logs in real time. Because I work with IIS and web applications, there are times when I have to troubleshoot incoming/outgoing IIS requests. Here is where this tool becomes handy. It has features in which you can highlight important lines. This option can be easily  configured by clicking on the “Highlighting” button from the upper menu. You can configure Bartail to search for a particular string and highlight it when it is logged:
Baretail utility
In this example, I’ve configured Bartail to search for “GET” requests coming on the server. You can configure background and font colors, set the Bold or Italic options and so on. Note that you can also add multiple lines highlighted differently to view them when parsing logs with many lines. By clicking the “Follow Tail” check box you can set the tool to follow each line as they are logged. Now click on Preferences > Options to view other available options:
Baretail
I’ve used this tool a lot when working with logs that are few GB and I would recommend it to anyone.
   Tail.exe – command line tool that is similar to Bartail. This utility can be used by downloading “Windows Server 2003 Resource Kit Tools” from Microsoft’s website. I think this tool was original available on Linux platforms and was later implemented for Windows OS. It has fewer options than Bartail and no GUI:
Tail command
This utility can only be used when following incoming and outgoing requests in real time.
   GVim – freeware tool, can be download from here. This utility is available in both installation and portable modes. I usually use the portable version because it is easier to manipulate when working with multiple machines. This tool comes with a graphical interface and has may configurable options. Because GVim has features for basic and complex parsing operations, I think that it provides all the things necessary for this action. The GUI is user friendly and intuitive:
GVim tool
   These are the tools that came to my mind. If you think there are other tools that worth mentioning don’t hesitate to leave a comment. Enjoy your day and stay tuned for the following article from IT training day.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s