This paper considers a sequence of stochastic dynamic problems in which the discount rate goes to zero. Sufficient conditions are given under which the limit of discounted optimal policies (and normalized values) are a (i) long-run average optimal and (ii) catching-up optimal policy (and value). These conditions are shown to be satisfied in many economic models. A new decision criterion for the undiscounted problem, the strong long-run average, is introduced. This criterion refines the long-run average, is often equivalent to the catching-up, and is easier to use than the latter since it has a recursive representation. A generalization of Fatou's lemma, which has wider applicability, is also proved. © 1991.