Using forward accumulation for automatic differentiation of implicitly-defined functions

This paper deals with the calculation of partial derivatives (w.r.t. the independent variables, x) of a vector of dependent variables y which satisfy a system of nonlinear equations g(u(x),y) = 0. A number of authors have suggested that the forward accumulation method of automatic differentiation can be applied to a suitable iterative scheme for solving the nonlinear system with a view to giving simultaneous convergence both to the correct value y and also to its Jacobian matrix y(x). It is known, however, that convergence of the derivatives may not occur at the same rate as the convergence of the y values. In this paper we avoid both the difficulty and the potential cost of iterating the gradient part of the calculation to sufficient accuracy. We do this by observing that forward accumulation need only be applied to the functions g after the dependent variables, y, have been computed in standard real arithmetic using any appropriate method. This so-called Post-Differentiation (PD) technique is shown, on a number of examples, to have an advantage in terms of both accuracy and speed over approaches where forward accumulation is applied over the entire iterative process. Moreover, the PD technique can be implemented in such a way as to provide a friendly interface for non-specialist users.