Given a European derivative security with an arbitrary payoff function and a corresponding set of underlying securities on which the derivative security is based. we solve the optimal-replication problem: Find a self-financing dynamic portfolio strategy-involving only the underlying securities-that most closely approximates the payoff function at maturity. By applying stochastic dynamic programming to the minimization of a mean-squared error loss function under Markov-state dynamics. we derive recursive expressions for the optimal-replication strategy that are readily implemented in practice. The approximation error or "epsilon" of the optimal-replication strategy is also given recursively and may be used to quantify the "degree" of market incompleteness. To investigate the practical significance of these epsilon -arbitrage strategies, we consider several numerical examples, including path-dependent options and options on assets with stochastic volatility and jumps.