A machine learning approach for Arabic text classification using N-gram frequency statistics

In this paper a machine learning approach for classifying Arabic text documents is presented. To handle the high dimensionality of text documents, embeddings are used to map each document ( instance) into R ( the set of real numbers) representing the tri-gram frequency statistics profiles for a document. Classification is achieved by computing a dissimilarity measure, called the Manhattan distance, between the pro. le of the instance to be classified and the profiles of all the instances in the training set. The class ( category) to which an instance ( document) belongs is the one with the least computed Manhattan measure. The Dice similarity measure is used to compare the performance of method. Results show that tri-gram text classification using the Dice measure outperforms classification using the Manhattan measure. (C) 2008 Elsevier Ltd. All rights reserved.