BotCatch: leveraging signature and behavior for bot detection

The goal of bot detection is to discover malicious bot processes by signature comparison or behavior analysis. Existing approaches have several drawbacks, such as requiring a lot of prior knowledge, low detection accuracy, and high false alarm rate. In this paper, we propose a multi-feedback approach, BotCatch, to detect bots effectively and efficiently on a host by leverage of a combination of signature and behavior. First, BotCatch assigns suspicious files to signature-analysis and behavior-analysis modules, which generate each detection result. Second, BotCatch correlates signature and behavior results to generate the final detection result through correlation engine. Third, BotCatch feeds back signature, behavior, and correlation results to dynamically adjust detecting modules through multi-feedback engine. We evaluated the performance of BotCatch with 636 bot and 150 benign samples. Our results indicate that BotCatch achieves an accuracy of 97.1% and an F-measure value of 0.982 simultaneously, which is better than existing approaches without feedbacks. BotCatch, due to the multi-feedback mechanism, has the ability to gradually get more robust and accurate as the number of samples increases. The final stage even reaches an accuracy of 98.5% and F-measure value of 0.991. Copyright (C) 2014 John Wiley & Sons, Ltd.