On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation

The fixed-interval Kalman smoother produces optimal estimates of the state of a system over a time interval, given observations over the interval, together with a prior estimate of the state and its error covariance at the beginning of the interval. At the end of the interval, the Kalman smoother estimate is identical to that produced by a Kalman filter, given the same observations and the same initial state and covariance matrix. For an imperfect model, the model error term in the covariance evolution equation acts to reduce the dependence of the estimate on observations and prior states that are well separated in time. In particular, if the assimilation interval is sufficiently long, the estimate at the end of the interval is effectively independent of the state and covariance matrix specified at the beginning of the interval. In this case, the Kalman smoother provides estimates at the end of the interval that are identical to those of a Kalman filter that has been running indefinitely. For a linear model, weak-constraint four-dimensional variational data assimilation (4D-Var) is equivalent to a fixed-interval Kalman smoother. It follows that, if the assimilation interval is made sufficiently long, the 4D-Var analysis at the end of the assimilation interval will be identical to that produced by a Kalman filter that has been running indefinitely. The equivalence between weak-constraint 4D-Var and a long-running Kalman filter is demonstrated for a simple analogue of the numerical weather-prediction (NWP) problem. For this nonlinear system, 4D-Var analysis with a 10-day assimilation window produces analyses of the same quality as those of an extended Kalman filter. It is demonstrated that the current ECMWF operational 4D-Var system retains a memory of earlier observations and prior states over a period of between four and ten days, suggesting that weak-constraint 4D-Var with an analysis interval in the range of four to ten days may provide a viable algorithm with which to implement an unapproximated Kalman filter. Whereas assimilation intervals of this length are unlikely to be computationally feasible for operational NWP in the near future, the ability to run an unapproximated Kalman filter should prove invaluable for assessing the performance of cheaper, but suboptimal, alternatives.