Kalman Filter

The Kalman filter (also known in control theory as Linear–Quadratic Estimation, or LQE) is a recursive mathematical algorithm in computation and algorithms that estimates the internal state vector of a time-evolving dynamical system from a time series of indirect, noise-corrupted measurements. Published in discrete-time form by Hungarian-American electrical engineer Rudolf E. Kálmán in 1960—and extended to continuous-time differential systems by Kálmán and Richard S. Bucy in 1961, building on earlier statistical foundations by Thorvald Nicolai Thiele (1880) and Peter Swerling (1958)—the filter operates in a two-phase predict–update cycle. Unlike Wiener filtering, which requires storing the entire historical observation record, the Kalman filter summarizes all past history within a sufficient statistic consisting of the current state estimate x̂_k|k and its n × n error covariance matrix P_k|k. When process disturbances and sensor errors follow Gaussian distributions, the Kalman filter implements recursive Bayesian Inference in closed form; when noise distributions are non-Gaussian with finite second moments, it remains the Best Linear Unbiased Estimator (BLUE).

State-Transition and Observation Equations

Consider a discrete-time linear dynamical system whose hidden state at time step k ∈ N is represented by x_k ∈ Rⁿ. The state transition from step k − 1 to k and the generation of an m-dimensional sensor measurement z_k ∈ R^m follow the coupled linear stochastic difference equations:

• State-Transition Equation: x_k = F_k x_k−1 + B_k u_k + w_k
• Observation Equation: z_k = H_k x_k + v_k

where F_k ∈ R^n×n is the state-transition matrix, B_k ∈ R^n×l is the control-input matrix acting on deterministic control vector u_k ∈ R^l, and H_k ∈ R^m×n is the observation matrix. The vectors w_k ∼ N(0, Q_k) and v_k ∼ N(0, R_k) represent mutually independent, zero-mean white Gaussian process noise (covariance Q_k ≥ 0) and measurement noise (covariance R_k > 0).

Phase 1: A Priori Time Update (Prediction)

Given the previous posterior state estimate x̂_k−1|k−1 and error covariance P_k−1|k−1 = E[(x_k−1 − x̂_k−1|k−1)(x_k−1 − x̂_k−1|k−1)^T], the filter projects the state and uncertainty forward across timestep k:

• Prior State Estimate: x̂_k|k−1 = F_k x̂_k−1|k−1 + B_k u_k
• Prior Error Covariance: P_k|k−1 = F_k P_k−1|k−1 F_k^T + Q_k

Phase 2: A Posteriori Measurement Update (Correction)

When measurement z_k arrives, the filter compares it against the predicted observation H_kx̂_k|k−1 to form the innovation (measurement residual) ỹ_k and its covariance S_k:

• Innovation Vector: ỹ_k = z_k − H_k x̂_k|k−1, with S_k = H_k P_k|k−1 H_k^T + R_k
• Optimal Kalman Gain: K_k = P_k|k−1 H_k^T S_k⁻¹
• Posterior State & Covariance: x̂_k|k = x̂_k|k−1 + K_k ỹ_k, and P_k|k = (I − K_k H_k) P_k|k−1

Optimal Gain Derivation and Joseph Stabilized Form

For any gain matrix K_k, expanding the covariance of the posterior error e_k = (I − K_kH_k)(x_k − x̂_k|k−1) − K_kv_k yields the symmetric Joseph stabilized covariance form:

P_k|k = (I − K_k H_k) P_k|k−1 (I − K_k H_k)^T + K_k R_k K_k^T

Differentiating the mean-squared error E[‖e_k‖₂²] = Tr(P_k|k) with respect to K_k and setting ∂Tr(P_k|k)/∂K_k = −2(H_kP_k|k−1)^T + 2K_kS_k = 0 yields K_k = P_k|k−1H_k^TS_k⁻¹. When sensor noise is large (R_k → ∞), K_k → 0 and the filter trusts its dynamical prediction; when prior uncertainty dominates (R_k → 0), K_k → H_k⁻¹ and the filter trusts the sensor measurement. To prevent floating-point loss of positive-definiteness, spacecraft navigation software uses the Joseph form or Potter's square-root filter, which propagates the Cholesky Decomposition factor P = LL^T.

Extended, Unscented, and Ensemble Kalman Filters

Physical systems in orbital mechanics, robotics, and geophysical fluid dynamics follow nonlinear state transitions x_k = f(x_k−1, u_k) + w_k and observation mappings z_k = h(x_k) + v_k:

Apollo Navigation and Geophysical Data Assimilation

The first operational deployment of the Kalman filter occurred at NASA's Ames Research Center under Stanley F. Schmidt in 1960–1961 for the Apollo Guidance Computer (AGC), fusing onboard sextant star-landmark angles with ground radar along the nonlinear Earth–Moon trajectory via the Extended Kalman Filter (EKF). In contemporary atmospheric and ocean sciences, Geir Evensen's Ensemble Kalman Filter (EnKF) (1994) assimilates millions of satellite radiance and ocean float observations into global forecast models with n > 10⁸ state variables.

Why is the Kalman filter called a recursive estimator?

Because the prior estimate x̂_k|k−1 and covariance P_k|k−1 form a sufficient statistic for all past measurements z_1:k−1, raw historical observations can be discarded after each update, keeping memory and CPU cost constant per timestep.

How does the Unscented Kalman Filter (UKF) improve upon the Extended Kalman Filter (EKF)?

The EKF linearizes f(x) via first-order Taylor Jacobians at the mean, which can underestimate covariance and diverge under strong curvature. The UKF (Julier and Uhlmann, 1997) propagates 2n + 1 deterministic sigma points through the true nonlinear function f, capturing the posterior mean and covariance to third order without Jacobians.

What is the difference between Kalman filtering and Kalman smoothing?

Filtering estimates the current state x_k given measurements up to the present (z_1:k), whereas smoothing (such as the Rauch–Tung–Striebel smoother) runs a backward pass over a recorded trajectory z_1:T to estimate x̂_k|T using both past and future observations.

1. Kálmán, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1), 35–45. https://doi.org/10.1115/1.3662552
2. Kálmán, R. E., & Bucy, R. S. (1961). New results in linear filtering and prediction theory. Journal of Basic Engineering, 83(1), 95–108. https://doi.org/10.1115/1.3658902
3. Rauch, H. E., Tung, F., & Striebel, C. T. (1965). Maximum likelihood estimates of linear dynamic systems. AIAA Journal, 3(8), 1445–1450. https://doi.org/10.2514/3.3166
4. Evensen, G. (1994). Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research: Oceans, 99(C5), 10143–10162. https://doi.org/10.1029/94JC00572
5. Julier, S. J., & Uhlmann, J. K. (2004). Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3), 401–422. https://doi.org/10.1109/JPROC.2003.823141