Evolutionary and dynamic stability in continuous population games

Asymptotic stability under the replicator dynamics over a continuum of pure strategies is shown-to crucially depend on the choice of topology over the space of mixed population strategies, namely probability measures over the real line. Thus, Strong Uninvadability, proved by Bomze (1990) to be a sufficient condition for asymptotic stability under the topology of variational distance between probability measures, implies convergence to fixation over a pure strategy x* only when starting from a population strategy which assigns to x* a probability sufficiently close to one. It does not imply convergence to x* when starting from a distribution of small deviations from x*, regardless of how small these deviations are. It is, therefore, suggested that when a metric space of pure strategies is involved, another topology, hence another stability condition, may prove more relevant to the process of natural selection. Concentrating on the case of a one dimensional continuous quantitative trait, we resort to the natural Maximum Shift Topology in which an epsilon-vicinity of the fixation on a pure strategy x* consists of all mixed population strategies with support which includes x* and is in the epsilon-neighborhood of x*. Under this topology, a relatively simple necessary and sufficient condition for replicator asymptotic stability, namely Continuous Replicator Stability (CRSS), is demonstrated. This condition is closely related to the static stability condition of Neighbor Invadability (Apaloo 1997), and slightly stronger than the condition of Continuous Stability (Eshel and Motro, 1981).