Learning in manifolds: The case of source separation

The blind signal separation (BSS) problem has a distinctive feature: the unknown parameter being an invertible matrix, the parameter set is a multiplicative group and the observations can be modeled by a transformation model. For this reason, it is possible to design on-line algorithms which are very simple and still offer excellent performance (typically: Newton-like performance at a gradient-like cost). This paper presents two apparently different approaches to deriving these algorithms from the maximum likelihood principle. One approach (relative gradient) starts with focus on the group structure and eventually introduces the statistical structure. The other approach (natural gradient) applies to any statistical manifold and is eventually made tractable by exploiting the group structure. The relationship between these approaches is explained.