On the optimality of conditional expectation as a Bregman predictor

We consider the problem of predicting a random variable X from observations, denoted by a random variable Z. It is well known that the conditional expectation E[X vertical bar Z] is the optimal L-2 predictor (also known as "the least-mean-square error" predictor) of X, among all (Borel measurable) functions of Z. In this correspondence, we provide necessary and sufficient conditions for the general loss functions under which the conditional expectation is the unique optimal predictor. We show that E[X vertical bar Z] is the optimal predictor for all Bregman loss functions (BLFs), of which the L-2 loss function is a special case. Moreover, under mild conditions, we show that the BLFs are exhaustive, i.e., if for every random variable X, the infimum of E[F(X, y) over all constants y is attained by the expectation E[X], then F is a BL.