Stationary iterative methods are methods for solving a linear system of equations

where  is a given matrix and  is a given vector. Stationary iterative methods can be expressed in the simple form

where neither  nor  depends upon the iteration count . The four main stationary methods are the Jacobi method, Gauss-Seidel method, successive overrelaxation method (SOR), and symmetric successive overrelaxation method (SSOR).

The Jacobi method is based on solving for every variable locally with respect to the other variables; one iteration corresponds to solving for every variable once. It is easy to understand and implement, but convergence is slow.

The Gauss-Seidel method is similar to the Jacobi method except that it uses updated values as soon as they are available. It generally converges faster than the Jacobi method, although still relatively slowly.

The successive overrelaxation method can be derived from the Gauss-Seidel method by introducing an extrapolation parameter . This method can converge faster than Gauss-Seidel by an order of magnitude.

Finally, the symmetric successive overrelaxation method is useful as a preconditioner for nonstationary methods. However, it has no advantage over the successive overrelaxation method as a stand-alone iterative method.

REFERENCES:

Barrett, R.; Berry, M.; Chan, T. F.; Demmel, J.; Donato, J.; Dongarra, J.; Eijkhout, V.; Pozo, R.; Romine, C.; and van der Vorst, H. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, 2nd ed. Philadelphia, PA: SIAM, 1994. http://www.netlib.org/linalg/html_templates/Templates.html.

Hageman, L. and Young, D. Applied Iterative Methods. New York: Academic Press, 1981.

Varga, R. Matrix Iterative Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1962.

Young, D. Iterative Solutions of Large Linear Systems. New York: Academic Press, 1971.