S-TRACE
It seems like in most real-world cases, log analysis is triggered by some stimuli, e.g. an alert (IDS, SIM, human) or a log report (text or graphical format) showing something interesting. Most sysadmins are probably too busy to consciously go and review logs unless something happens.
It also seems like most of the time, the process of investigation is in reverse chronological (temporal) order, as in, you go backwards in order to find out what happened (when did the attack occur, where did it come from, how did it occur). Sometimes you will go forward in time to find out if other similar incidents has happened or if the attacker used the host for other attacks.
There’s also a spatial component to the process of correlation as well. We generally look at logs from multiple hosts/devices across the network in order to figure out the root cause.
Almost all of the SIM vendors are implementing their systems based on forward temporal correlation, this is probably useful in detecting some of the more well known scenarios. If the scenario is not known, most likely the attack will be missed.
For most investigations and root cause analysis, it seems like reverse spatial & temporal analysis is more useful. Also, it might be less processor and memory intensive.
I call this S-TRACE, Spatial & Temporal Reverse Analytics & Correlation Engine.
The question is can something like that built to perform the reverse correlation automatically (or if it’s useful at all). There probably has to be some rules to tell the engine what to look at next (in reverse).
I would love to hear your thoughts on this.