Posted on

Lanczos algorithm

Lanczos algorithm

From Wikipedia, the free encyclopedia
The Lanczos algorithm is an iterative algorithm invented by Cornelius Lanczos that is an adaptation of power methods to findeigenvalues and eigenvectors of a square matrix or the singular value decomposition of a rectangular matrix. It is particularly useful for finding decompositions of very large sparse matrices. In Latent Semantic Indexing, for instance, matrices relating millions of documents to hundreds of thousands of terms must be reduced to singular-value form.
Peter Montgomery published in 1995 an algorithm, based on the Lanczos algorithm, for finding elements of the nullspace of a large sparse matrix over GF(2); since the set of people interested in large sparse matrices over finite fields and the set of people interested in large eigenvalue problems scarcely overlap, this is often also called the block Lanczos algorithm without causing unreasonable confusion. See Block Lanczos algorithm for nullspace of a matrix over a finite field.
Posted on

Arnoldi iteration

Arnoldi iteration

From Wikipedia, the free encyclopedia
In numerical linear algebra, the Arnoldi iteration is an eigenvalue algorithm and an important example of iterative methods. Arnoldi finds the eigenvalues of general (possibly non-Hermitian) matrices; an analogous method for Hermitian matrices is the Lanczos iteration. The Arnoldi iteration was invented by W. E. Arnoldi in 1951.
The term iterative method, used to describe Arnoldi, can perhaps be somewhat confusing. Note that all general eigenvalue algorithms must be iterative. This is not what is referred to when we say Arnoldi is an iterative method. Rather, Arnoldi belongs to a class of linear algebra algorithms (based on the idea of Krylov subspaces) that give a partial result after a relatively small number of iterations. This is in contrast to so-called direct methods, which must complete to give any useful results.
Arnoldi iteration is a typical large sparse matrix algorithm: It does not access the elements of the matrix directly, but rather makes the matrix map vectors and makes its conclusions from their images. This is the motivation for building the Krylov subspace.
Posted on

Krylov subspace

Krylov subspace

From Wikipedia, the free encyclopedia
In linear algebra, the order-r Krylov subspace generated by an n-by-n matrix A and a vector b of dimension n is the linear subspacespanned by the images of b under the first r powers of A (starting from A0 = I), that is,
mathcal{K}_r(A,b) = operatorname{span} , { b, Ab, A^2b, ldots, A^{r-1}b }. ,
It is named after Russian applied mathematician and naval engineer Alexei Krylov, who published a paper on this issue in 1931.[1]
Modern iterative methods for finding one (or a few) eigenvalues of large sparse matrices or solving large systems of linear equations avoid matrix-matrix operations, but rather multiply vectors by the matrix and work with the resulting vectors. Starting with a vector, b, one computes Ab, then one multiplies that vector by A to find A2b and so on. All algorithms that work this way are referred to as Krylov subspace methods; they are among the most successful methods currently available in numerical linear algebra.
Because the vectors tend very quickly to become almost linearly dependent, methods relying on Krylov subspace frequently involve some orthogonalization scheme, such as Lanczos iteration for Hermitian matrices or Arnoldi iteration for more general matrices.
The best known Krylov subspace methods are the Arnoldi, Lanczos, Conjugate gradient, GMRES (generalized minimum residual),BiCGSTAB (biconjugate gradient stabilized), QMR (quasi minimal residual), TFQMR (transpose-free QMR), and MINRES (minimal residual) methods.

References

  1. ^ Mike Botchev (2002). “A.N.Krylov, a short biography”.
Posted on

Successive over-relaxation

Successive over-relaxation

From Wikipedia, the free encyclopedia
In numerical linear algebra, the method of successive over-relaxation (SOR) is a variant of the Gauss–Seidel method for solving alinear system of equations, resulting in faster convergence. A similar method can be used for any slowly converging iterative process. It was devised simultaneously by David M. Young and by H. Frankel in 1950 for the purpose of automatically solving linear systems on digital computers. Over-relaxation methods had been used before the work of Young and Frankel. For instance, the method of Lewis Fry Richardson, and the methods developed by R. V. Southwell. However, these methods were designed for computation by human calculators, and they required some expertise to ensure convergence to the solution which made them inapplicable for programming on digital computers. These aspects are discussed in the thesis of David M. Young.[1]
Posted on

Jacobi method

Jacobi method

From Wikipedia, the free encyclopedia
In numerical linear algebra, the Jacobi method is an algorithm for determining the solutions of a system of linear equations with largest absolute values in each row and column dominated by the diagonal element. Each diagonal element is solved for, and an approximate value plugged in. The process is then iterated until it converges. This algorithm is a stripped-down version of the Jacobi transformation method of matrix diagonalization. The method is named after German mathematician Carl Gustav Jakob Jacobi.
Posted on

Metzler matrix

 a Metzler matrix is a matrix in which all the off-diagonal components are nonnegative (equal to or greater than zero)
qquad forall_{ineq j}, x_{ij} geq 0.
Metzler matrices appear in stability analysis of time delayed differential equations and positive linear dynamical systems. Their properties can be derived by applying the properties of Nonnegative matrices to matrices of the form M + aI where M is a Metzler matrix.
Posted on

P-matrix

a P-matrix is a complex square matrix with every principal minor > 0. A closely related class is that of P0-matrices, which are the closure of the class of P-matrices, with every principal minor geq 0.


Spectra of P-matrices

By a theorem of Kellogg, the eigenvalues of P– and P0– matrices are bounded away from a wedge about the negative real axis as follows:
If {u1,…,un} are the eigenvalues of an n-dimensional P-matrix, then

|arg(u_i)| < pi - frac{pi}{n}, i = 1,...,n
If {u1,…,un}, u_i neq 0, i = 1,…,n are the eigenvalues of an n-dimensional P0-matrix, then

|arg(u_i)| leq pi - frac{pi}{n}, i = 1,...,n

Notes

The class of nonsingular M-matrices is a subset of the class of P-matrices. More precisely, all matrices that are both P-matrices and Z-matrices are nonsingular M-matrices.
If the Jacobian of a function is a P-matrix, then the function is injective on any rectangular region of mathbb{R}^n.
A related class of interest, particularly with reference to stability, is that of P( − )-matrices, sometimes also referred to as N − P-matrices. A matrix A is a P( − )-matrix if and only if ( − A) is a P-matrix (similarly for P0-matrices). Since σ(A) = − σ( − A), the eigenvalues of these matrices are bounded away from the positive real axis.


References

  • R. B. Kellogg, On complex eigenvalues of M and P matrices, Numer. Math. 19:170-175 (1972)
  • Li Fang, On the Spectra of P– and P0-Matrices, Linear Algebra and its Applications 119:1-25 (1989)
  • D. Gale and H. Nikaido, The Jacobian matrix and global univalence of mappings, Math. Ann. 159:81-93 (1965)
Posted on

Z-matrix

 the class of Z-matrices are those matrices whose off-diagonal entries are less than or equal to zero; that is, a Z-matrix Z satisfies
Z=(z_{ij});quad z_{ij}leq 0, quad ineq j.
Note that this definition coincides precisely with that of a negated Metzler matrix or quasipositive matrix, thus the term quasinegative matrix appears from time to time in the literature, though this is rare and usually only in contexts where references to quasipositive matrices are made.
The Jacobian of a competitive dynamical system is a Z-matrix by definition. Likewise, if the Jacobian of a cooperative dynamical system is J, then (−J) is a Z-matrix.
Related classes are L-matrices, M-matrices, P-matrices, Hurwitz matrices and Metzler matrices. L-matrices have the additional property that all diagonal entries are greater than zero. M-matrices have several equivalent definitions, one of which is as follows: a Z-matrix is an M-matrix if it is nonsingular and its inverse is nonnegative. All matrices that are both Z-matrices and P-matrices are nonsingularM-matrices.
Posted on

M-matrix

 An M-matrix is a Z-matrix with eigenvalues whose real parts are positive. M-matrices are a subset of the class of P-matrices, and also of the class of inverse-positive matrices (i.e. matrices with inverses belonging to the class of positive matrices).[1]
A common characterization of an M-matrix is a non-singular square matrix with non-positive off-diagonal entries and all principal minors positive, but many equivalences are known. The name M-matrix was seemingly originally chosen by Alexander Ostrowski in reference to Hermann Minkowski.[2]
A symmetric M-matrix is sometimes called a Stieltjes matrix.
M-matrices arise naturally in some discretizations of differential operators, particularly those with a minimum/maximum principle, such as the Laplacian, and as such are well-studied in scientific computing.
The LU factors of an M-matrix are guaranteed to exist and can be stably computed without need for numerical pivoting, also have positive diagonal entries and non-positive off-diagonal entries. Furthermore, this holds even for incomplete LU factorization, where entries in the factors are discarded during factorization, providing useful preconditioners for iterative solution.