The clustering approach to malware dataset analysis

Abstract

The research in the area of malware analysis is very popular with accent on machine learning algorithms that help automate this subject. One of the leader portals that help researchers with dataset problems is Virus Total, providing free academic accounts with hundreds of thousands malware samples with metadata. This work contributes with the analysis of 429058 malware samples from Virus Total in terms of overcoming the problem of inconsistent labeling of the antivirus scan results from different vendors. Two methods were used, LSA and LDA both with automatic calibration of parameters, with purpose of finding the optimal number of clusters - both resulting with 5. The graphical representation of the clusters was done by k-menas clustering in two dimensional space. Additional research on most informative words in each cluster, showed that we can report of 4 similar classes, and one cluster per method (LSA and LDA) that was not related to the cluster in opposite method by wrd meaning in it. The showed results give good approach malware data analysis when dealing with inconsistently labeled dataset.