Zen 2.0

Most of the logs generated in a corporate infrastructure are not reviewed. They are either archived and never looked at, or worse, never even retrieved and archived. Much of the logs generated by devices and applications evaporate into the ether and not missed.

There are many reasons why most of these logs and events are ignored.

First of all, the volume of logs generated by the infrastructure is ENORMOUS. Log volumes can easily go from megabytes to tens of gigabytes. Sifting through the logs and trying to analyze them is such a huge task that no sane human being would put herself through that torture.

Secondly, most of the time the administrators are not sure what to look for. Are they suppose to look for security problems? If so, what kind of security problems? What is considered to be a security breach? Are they suppose to use the data to gather operational intelligence? If so, what should they be looking for?

Thirdly, building a logging infrastructure is not a simple task. It requires a lot of planning. For example, what logs to capture, how to capture them, where to store them, how long to store them, all these questions have to be answered. With tight IT budgets and few IT administrators, many organizations are pushing the problem off until they have more resources (and budget).

Last but not least, some organizations don’t see the value in reviewing logs. They believe that they can get their information via other means. For example, they believe that security intelligence can come from their IDS boxes; they believe that IT intelligence can come from their ESM products; and they believe that business intelligence can come from their BI applications.

All of the reasons are valid and I am definitely not trying to debunk them. However, more and more organizations are seeing the true value of log analysis and are jumping on board. Better technology and lower TCO will also persuade some of the doubters to test the water.