Zen 2.0

Flew over here to sunny PHX on Sunday to attend the DEMO 2006 conference, the leading conference for launching new products and services.

It’s my first time attending the conference and it’s interesting to see how organized they want you to be when doing a demo. You must have the whole demo script ready and sent in days before the conference. You have either a 3 or 6-minute slot for the demo. Everything’s well scripted and well rehearsed.

Will try to blog a few interesting items I see over here…

p.s. LogLogic’s launching 3 r2, the latest release. Check out our website.

wow…this is really kewl!

It’s a terminal program for OS X that emulates to old amber or green terminals that I used back at Purdue!!

Running it on full screen on a small monitor would really make this a pretty awesome setup!

A vulnerability has been reported in Cisco Security Monitoring, Analysis and Response System (CS-MARS), which can be exploited by malicious, local users to gain escalated privileges.

The vulnerability is caused due to CS-MARS appliances being shipped with a default password for an undocumented administrative root account that is intended for debugging purposes. This can be exploited by malicious users to gain root privileges using the undocumented “expert” command. The password for the account reportedly cannot be changed.

Successful exploitation requires logon to the administration command line interface with e.g. the “pnadmin” account.

The vulnerability has been reported in versions prior to 4.1.3.

Please upgrade to 4.1.3 asap.

As reported by Secunia and Cisco.

Found this interesting quote in the abstract of Leveraging Operational Intelligence - The Battle of Tannenberg and Masurian Lakes (1914)

A commander’s ability to leverage operational intelligence remains a cornerstone for managing the uncertainty, fog, and friction of war in this era of seemingly endless advances in battlefield awareness and information dominance. Operational intelligence, properly understood and utilized, can become a force multiplier maximizing traditional operational art factors. The goal is information superiority thereby increasing the speed at which decisions are made, while preempting enemy choices and courses of action.

**CAREER FAIR**
Wednesday, January 11th at 4-8pm
LogLogic, Inc., 3061-B Zanker Road, San Jose, CA 95134
www.loglogic.com

We are hiring! Interview with Hiring Managers & VP’s for the following positions:

Loglogic is a premier start up, funded by leading VC firms (Sequoia Capital, Telesoft Partners, Worldview) with a broad range of both domestic and international customers. Our customers include Fortune 500 corporations, leading financial and healthcare service providers and managed services enterprises, to name a few. Our company develops ultra high performance Log management appliances on Linux platforms for enterprise IT departments to meet compliance and legal requirements. We accelerate and support responses to data center security and performance events, while providing complete log data archives for compliance and legal protection. Please visit us if you enjoy the startup environment, with its high demands and high rewards.

• Senior Computer Scientist
• Sr. Database Engineer, Search & Semi-Structured
• Sr. Software Engineer, Parsers and Compilers
• Sr. Indexing/Lucene Engineer
• Sr. Web Application Engineer
• Software Engineers (C, C++, Java)
• IT Lab Administrator
• Sr. Tech Writer
• Director, Product Management

LogLogic’s ST appliances were named winner of a Best of Interop Award at NetWorld+Interop 2005 Las Vegas.

Bring your resume or paste your resume into an email with “POSITION –YOUR NAME” in the subject line to: careers@loglogic.com, Attn: V. Golub. Please visit us at www.loglogic.com or our blog at blog.loglogic.com.

A very interesting and useful discussion took place the last week on the LogAnalysis mailing list.

Anton Chuvakin started the thread by asking other than parsing the individual messages (that could potentially have thousands of different formats), what other methods can be used in analyzing logs?

Some suggestions out of this discussion are listed here.

Clustering

Anton listed this as an option using tools such as slct. Another effort that I am aware of that’s using this approach is Securimine for Snort (SFS) from Securimine.

Securimine is founded by Ophir Rachman, who also founded Entercept Security Technologies (later on acquired by McAfee).

Brute-force Parsing

This method basically tries to guess some of the data structures inside a log message, such as IP address, hostname, username, password, action, etc etc.

Being able to correctly guess what data is a message without first knowing the message format is a tough problem. It relies on the parser knowing the exact structure of some of the data.

However, it can still be used to assist in parsing unknown messages. You can also apply some simple logics to classify the messages. Such as, if you see keywords such as from or to and IP addresses, that may be a firewall message.

Obviously this is not a fool-proof way, but given the alternative (not doing anything with the message at all!), it is a viable solution.

(One may ask the question of, is it better to not do anything so the users won’t be misled? or is it better to attempt in guessing and possibly give the wrong information? what do you think?)

Bayes/Markov/Expert Systems/Neural Nets/Genetic Algorithms

Several of the statisitical type of analysis were mentioned here.

1. Expert system - a collection of empirical data and decision algorithms compiled by developers
2. Hidden Markov models - since they are used in natural language and speech processing they might be applicable to log entries (they are after all some type of “natural speech”).
3. Neural nets - Once built, the neural net would be trained by experienced teachers (log analysis gurus).
4. Genetic algorithms - The trick would be to 1. define the right requirements (for example, determine the least number of message types without discarding significant data) and 2. define the genetic codes for the solution organisms. Maybe GAs are a bit far fetched but I wouldn’t exclude them.
5. Bayes - Bayesian classifiers have been extremely popular and successful in spam filtering. The success of baysian in spam filtering is partly due to the simplicity of classifying emails into ham and spam. In the log world, it is much tougher to tell from good to bad. Also, lots of not-bad messages may also indicate something bad. So it is tough to say how one can apply this type of technology to log analysis.

Obviously I am no mathematician nor do I claim to understand the nitty-gritty details of statistical analysis, so I can’t comment much on the technical merit of these methods. But love to hear from anyone who have more knowledge.

Indexing

One of the newer methods of analyzing logs is indexing and providing Google like search capabilities for all logs. This is something LogLogic and Splunk are doing.

The basic idea is that instead of parsing the messages by understanding every single format, use the full-text indexing approaches to break the messages into tokens, then allow users to use boolean search expressions to search the logs.

This method is great when it comes to troubleshooting and forensic analysis. If complemented with the understanding of the log formats, it can be as powerful as other methods.

I wrote an article on Searching for Root Cause a while back on the benefit of using Google-like indexed search on logs.

Tokenizing

This is the way most log analyzers are using today. This method generally require writing regular expressions or similar methods to parse the individual pieces of information out of the log messages.

Rainer Gerhards has a great summary in his paper On the Nature of Syslog Data.

Various standards

About Windows Event Log

IBM’s Common Base Event XML format - This is a VERY complicated XML based format that tries to cover everything. I see two huge problem with this type of format. First, it hugely expands the storage requirement given that raw log storage is required. Second, it could make parsing that much slower given the size of a single log (multiple KBs instead of hundres of bytes). It’s been morphed into the OASIS standard WSDM Management Using Web
Services v1.0 (WSDM-MUWS) .

RFC 3881 - Security Audit and Access Accountability Message XML Data Definitions for Healthcare Applications

WELF

W3C

IDMEF - Intrusion Detection Message Exchange Format

IDIOM - Intrusion Detection Interaction and Operations Messages (Cisco message format)

Gallery 2.0.1 and 2.0 has a minor security flaw. Here’s from the gallery web site:

Gallery 2.0.2 is now available for download. This release adds no new features. It fixes a minor XSS exploit, a potential information leak and a file disclosure bug in the zipcart module that could allow remote visitors to view sensitive files on your webserver. These security flaws were discovered during an internal security audit of the Gallery 2 code, and there are no known exploits of them in the wild. However we strongly recommend that you upgrade to version 2.0.2 as soon as possible. If you’re unable to upgrade right away we recommend that you disable the zipcart module until time permits you to upgrade.

I came back today and saw a TON of access from various IPs. It is especially bad since now there seems to be an automated process that checks for this exploit. Ran the following to get the offending IPs:

tail -20000 access_log|grep ‘\.\.\.\.\.\.\/1\.0′|cut -f1 -d’ ‘|sort|uniq

The offending IPs seem to be:

• 12.44.172.92
• 12.44.181.220
• 63.160.77.236

It seems to have crawled the web for URLs that link to the gallery pictures and used those URLs to get to the gallery sites. It looks for both /album and /gallery URLs.

The logs are similar to

12.44.172.92 - - [04/Dec/2005:15:24:56 -0800] “GET /album/sa/ecuador/sa1.html HTTP/1.0″ 302 276 “-” “Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ……/1.0 )” “-”

or

63.160.77.236 - - [04/Dec/2005:15:24:28 -0800] “GET /gallery/main.php?g2_view=core.ShowItem&g2_itemId=12&
g2_GALLERYSID=21831e46358ea023c3289f30b9f7ffb5 HTTP/1.0″ 200 14830 “-” “Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ……/1.0 )” “-”

If you use those URLs, you would get something like

Notice the “System Information” section? It shows a ton of stuff about your setup.

After the upgrade, that whole section will be gone, giving only the “Error Detail” section.

So I have been doing quite a bit of international traveling, both business and personal. I am slowly finding out there is evil in credit card companies. For example, I have been using a Citi card for most of the charges. Every time the clerk swipes my card, 3% is added to the purchase amount. If I were to use an Amex, that’s an additional 2% on top of the purchase amount.

This is crazy!! It’s not like they have to do anything. When the bank receives the $$, it’s already in US$!!

In any case, here are some articles that explains the details:

• Uncovering credit card costs on foreign charges
• Look out for fees on foreign charges
• 7 rules for using your card overseas
• New credit card fees take a further swipe at dollar

There are a lot more articles on the web. You can search for it yourself.

In any case, most of these articles recommend getting a Capital One card that does not charge the extra fee. I think that’s what I’ll do and cancel the others.

Our recent trip to China included a press conference with many of the top newspapers and IT magazines in China. Here’s a press piece that appeared on DoSTOR.

在商业竞争异常激烈的今天，日志存储和数据处理方面的需求将不断增长，而且企业常常是将其作为基础设施来规划，日志管理已成为世界上IT管理的新趋势。因此，LogLogic公司率先提出“日志生命周期智能化管理”概念，对日志数据进行采集、汇聚、存储、归档、分析和报警、实时监控，这种有效的管理方法可以优化网络运行效果, 提高网络可用性和安全性；提供可靠的网络信息审计；简化为适应行业法规要求所需的工作，从而大大降低运营成本；还可以协助IT决策和风险管理。

Recently Chris, Andy and I went to visit China. Here are some photos Andy took.

My company, LogLogic, has officially launched LogLogic 3. Along with the new product launch, we also luanched
a new loglogic.com web site and a new blog, LogBlog.

The new web site looks awesome!! Hate to say this but it’s much better than the old one.

The blog is also a great resource for information on LogLogic as well as tips and hints on how to manage the enterprise log data.

Check’em out..

So as I was going back through this month’s news, it seemed that one of the biggest news is that Google is bidding to provide free wi-fi service to the city of San Francisco. The proposal is to blanket 49 square miles with up to 30 access points per square mile and will server over 800,000 users.

So what is up with Google? I mean, their mission is to organize the world’s information, not to build telecom infrastructures. Many of the reports out there seem to think this is an obvious and logical step for Google to take. Their rationale, since Google’s organizing the world’s information, it makes sense that they provide users free access to the information as well.

Well, it seems logical. But that just doesn’t seem like a good enough reason to spend tens of millions of dollars to build a free wi-fi network.

But if you buy into the thinking that “the network is your computer” thinking, this may actually make sense. All of the applications that Google’s developing, Gmail, Google Reader, etc, are web-based (fully ajaxed, web 2.0, name the term you want to use) applications. With the recent deal with Sun, there’s a lot of rumors going around that says Google will provide a web-based version of OpenOffice.

Even if Google can provide all these applications, how does Google get users to go use it? Majority of the users when they start their browser, they will go to msn.com by default. That’s not to say these users won’t just type in google.com in their browser, but Microsoft gets a lot of their traffic due to IE’s default home page being msn.com.

In order for Google to get majority of the users to use their web-ased applications, they have to change the default destination on these users’ browsers. The first step Google took was to introduce Google Desktop. It’s a great tool that allows you to search your computer as well as the web. However, the problem with Google Desktop is that users still have to go to Google, download the tool and install it. So Google Desktop alone just won’t do the job.

So what better way to take control of the desktop and change the default destination of all users than to give them free access? Free access will make sure their first web page is Google instead of MSN.

As they all say, free is good. Free access will pretty much guarantee that majority of the 800,000 users will use ite, even if it’s only 300Kbits. Google says they will work with other providers to provide fee-based higher speed connection.

Some estimate says that the cost to build such a network is around $10 million. That’s approximately $13 per user, which is an extremely low price for acquiring users.

So is Google’s wi-fi ambition really a step in competing with Microsoft for the control of the desktop? I really think so.

Oh my gawd, 3 posts in 1 day and no real content…wassup with that. I promise, when I get a chance one of these days, I will start generating some real content…

On to official business…

Now here is the catch. I can’t tell you to vote as according to the rules, no agent, employee, shareholder holding more than 5 per cent of a company’s stock, director or affiliate of the Nominee may vote for the Nominee’s products or services. But we do want to win!

So, you can really help by getting the word out! LogLogic is starting a grass-roots web effort to support our “Log Vote 2006″ campaign. LogLogic is nominated for the 2006 SC Magazine Awards in two categories. Tell your friends, family and customers. Get people to vote for LogLogic.

Best Security Audit

Best Computer Forensics

My company, LogLogic, is looking to fill a “log analyst” position. We are looking for someone who can

- understanding the log formats and transport mechanisms
- researching different log formats to identify common categories (to help design the back end)
- parsing and normalizing the logs for the necessary information, based on requirements
- mapping the reports to parsed information or vice versa
- identifying useful reports and alerts based on the log information

If you know of anyone who might fit the bill, please let them know and have them forward their resume to zhenjl(at)gmail.com