Davidon–Fletcher–Powell formula

Content sourced from Wikipedia, licensed under CC BY-SA 3.0.

The Davidon–Fletcher–Powell (DFP) formula is a method used in optimization to update the Hessian approximation in a way that makes it easier to find the minimum. It was the first quasi-Newton approach to extend the one-dimensional secant method to problems with many variables, and it keeps the Hessian estimate symmetric and positive definite.

At iteration k, after moving from x_k to x_{k+1}, we define s_k = x_{k+1} − x_k and y_k = ∇f(x_{k+1}) − ∇f(x_k). If the curvature condition s_k^T y_k > 0 holds, the DFP update for the Hessian B_k is

B_{k+1} = B_k + (y_k y_k^T) / (s_k^T y_k) − (B_k s_k s_k^T B_k) / (s_k^T B_k s_k).

This update makes B_{k+1} s_k = y_k, while keeping B_{k+1} symmetric and positive definite. The inverse Hessian H_k = B_k^−1 also has a corresponding dual update, and DFP can be written in a compact form using all past s and y vectors. This compact representation is especially useful for large-scale or constrained problems.

DFP was later largely superseded by the Broyden–Fletcher–Goldfarb–Shanno (BFGS) formula, which updates the inverse Hessian in a similar spirit but with different bookkeeping that often offers better numerical stability. In short, DFP provides a principled way to adjust the second-derivative model to match observed gradient changes while preserving key properties of the estimate.