Authentium Virus Blog

This will be my third blog posting about the dangers of multiscanners. The previous ones are: Malware Naming Confusion and Multiscanners: The good, the bad and the ugly.

This time it is about how garbage can spread and infect definition files.

What I am talking about is somebody is detecting some file just because somebody else is. Or people demand that we detect a file just because somebody else is.

We accidentally did a test that was similar to the one that Kaspersky did that lead to my first blog entry on this subject. The original test was that they created several executables and added detection for it. These executables were then submitted to a public multiscanning service and they monitored detection of these samples. After a few weeks a large number of vendors detected the samples with virtually identical names.

We have a file that we detect as W32/TestSample. It is a simple Windows executable that does 2 things. It opens a handle to itself, which always fails, and it displays a message box stating that it is a test sample. This allows our OEM customers to test our memory and registry scanning routines and to test the functionality of the scan engine in a safe environment. It definitely is not malware.

Or so I thought.

Obviously if 13 products detect a file it must be malware. Interestingly enough some of the major brands detect this file. I would love to name and shame, but I am confident there must be some similar garbage in our own collection. What is really funny is that only one other vendor at least tried to copy our name for this. Two other vendors called it Adware, 3 detected it heuristically and 6 others detected it accurately by name using different names.

This specific sample is at least in one large testing organizations test set that is used to determine malware detection rates.

It is easy to be critical towards these other companies for including detection for this but in the end I think I understand how it happened. Either a mutual client insisted that these samples must be detected or they saw the samples in the test set of some antivirus tester and decided to add detection for it because it is easier than to dispute the sample. Or they saw that we detected it and therefore added detection for it without analyzing the actual sample.

Just be careful how you interpret the results of a multiscanner. Not everything detected by antivirus products are really malware. There are too many non-technical influences on what should be detected for the results of a multiscanner to be valuable.

What is a multiscanner?

It is a system where multiple AV products are used to scan files and provide a report about the files.

What is the good about a multiscanner?

What is bad about a multiscanner?

Before I go on to the ugly I need to explain the previous statements. When a scanner detects something the only information that you have is that some scanner detected that file. It does not imply that the file is malware.

That may sound surprising but the reality is that not all scanners were created equally. Some scanners have heuristics that is so paranoid that they trigger on virtually every second file. Today I spent some time looking at about 200 files gathered by one of our monitoring systems. The files were from three different sources and about 90% of them were detected by at least one scanner. Strangely enough they were all variants of three totally legitimate products and should not have been detected. Had I trusted the multiscanner I would have duplicated their mistake.

Not all scanners are meant to be used this way. Some scanners are focused on scanning email or gateway based traffic. This enables them to specialize and tweak their heuristics in such a way that they do an amazing job at the gateway but would be disastrous to use on the desktop. They have virtually no false positives when used as they were designed, but when used in a multiscanner or as part of a desktop product they would behave in very unexpected ways.

Some scanners will automatically detect that it is being used to scan a collection of malware and change their behavior. This can make it difficult to trust in a multiscanner environment as its behavior is not consistent and the results can be surprising.

I am not going to name any specific vendors but completely trusting the information you get from a multiscanner is quite dangerous. The information gathered from a multiscanner about a file have to be added to other information about the file before a determination can be made whether the file is malware or not.

What is ugly about a multiscanner?

This article was triggered by a highly amusing article by Kaspersky: On the way to better testing. Responses by ESET and PC Magazine. What amused me is to know that they are in the same boat as we are. It is hard to convince a customer that some file that some so-called “tester” magically conjured up from some disreputable spot is garbage if 10 other scanners detect it. What is less amusing is if one of your own products are detected by 10 other products and you try to get it white listed or detection for it removed.

Both these situations are the real ugly of multiscanners. Not only the ones in use by every AV company out there but also the public ones.

I am not trying to point any fingers nor am I saying that any individual scanner or multiscanner should not be trusted. All I am saying is that the context in which a product or products are used should be understood. The risks and the value of the information provided by any source should be investigated and understood before a decision based on the information is made.

So there is this 18 year old kid that is making news. Some people call him a security professional, some people give him credit for “advancing the state of security”, some people even call him an antivirus researcher.

This is based on the fact that the 18 year old Peter Kleissner has started doing things that would not be considered good behavior in the antivirus industry. Some of it is documented at Former Anti-Virus Researcher Turns Tables On Industry.

You will have noticed that thus far I have mentioned his age twice. If you go to his blog you will notice that the school he is studying at seems to be investigating him for other behaviors too.

Lets get some balance to this story. I think he is doing a really good job at destroying himself and I hope he has friends and family that can help and support him when he figures that out. I am not trying to protect him or to say what he is doing has any merit, it does not.

What I am saying is that he is young, angry and confused. He is also just 18. He probably does not know enough yet to be a danger to anybody but himself and is probably about 10 years away from making a lasting impression in any field. Lets hope he figures out what he is doing to himself before he gets into real trouble.

Now that I have said nice things about him. I don’t think anything he has done is really going to damage the antivirus industry. I also think that Ikarus was totally correct in asking him to leave. If he was my employee I would probably have done the same thing.

I think the real malware groups out there are doing things much more professionally. They are using people that are paid more and better qualified than Peter. I personally would not recommend a job as a malware author either: I have a suspicion that side of the fence use baseball bats. We just fire people or force them to listen to rants about malware naming conventions.

I also think this shows that the antivirus industry is serious about staying on the right side of the fence. Due to that he will probably never work in the antivirus industry again and any “security” company that appoints him will announce something about their ethics.