Conditionally independent component analysis for supervised feature extraction

The present paper extends the framework of independent component analysis (ICA) to supervised learning. The key idea is to find a conditionally independent representation of input variables for a target variable by linear transformation. The representation can be considered as independent components of observations from which explanatory parts for a target variable are removed, and it is directly useful for naive Bayes learning which has been reported to perform as well as more sophisticated methods for prediction. The learning algorithm is derived under a similar but different criterion to ICA. The algorithm attempts to maximize the independence among extracted features as well as the mutual information between extracted features and a target variable. (C) 2002 Elsevier Science B.V. All rights reserved.