Matrix norm
In mathematics, a matrix norm is a vector norm in a vector space whose elements (vectors) are matrices (of given dimensions). Preliminaries 

Given a field  of either real or complex numbers, let  be the -vector space of matrices with  rows and  columns and entries in the field . A matrix norm is a norm on . This article will always write such norms with double vertical bars (like so: ). Thus, the matrix norm is a function  that must satisfy the following properties:

For all scalars  and matrices ,
 (positive-valued)
 (definite)
 (absolutely homogeneous)
 (sub-additive or satisfying the triangle inequality)

The only feature distinguishing matrices from rearranged vectors is multiplication. Matrix norms are particularly useful if they are also sub-multiplicative:

Every norm on  can be rescaled to be sub-multiplicative; in some books, the terminology matrix norm is reserved for sub-multiplicative norms. Matrix norms induced by vector norms

Suppose a vector norm  on  and a vector norm  on  are given. Any  matrix  induces a linear operator from  to  with respect to the standard basis, and one defines the corresponding induced norm or operator norm or subordinate norm on the space  of all  matrices as follows:

where  denotes the supremum. This norm measures how much the mapping induced by  can stretch vectors. Depending on the vector norms ,  used, notation other than  can be used for the operator norm. Matrix norms induced by vector p-norms
If the p-norm for vectors () is used for both spaces  and  then the corresponding operator norm is:

These induced norms are different from the "entry-wise" p-norms and the Schatten p-norms for matrices treated below, which are also usually denoted by 

In the special cases of  the induced matrix norms can be computed or estimated by

which is simply the maximum absolute column sum of the matrix;

which is simply the maximum absolute row sum of the matrix. For example, for

we have that

In the special case of  (the Euclidean norm or -norm for vectors), the induced matrix norm is the spectral norm. (The two values do not coincide in infinite dimensions — see Spectral radius for further discussion.) The spectral norm of a matrix  is the largest singular value of  (i.e., the square root of the largest eigenvalue of the matrix  where  denotes the conjugate transpose of ):

where  represents the largest singular value of matrix  Also,

since  and similarly  by singular value decomposition (SVD). There is another important inequality:

where  is the Frobenius norm. Equality holds if and only if the matrix  is a rank-one matrix or a zero matrix. This inequality can be derived from the fact that the trace of a matrix is equal to the sum of its eigenvalues. When  we have an equivalent definition for  as  It can be shown to be equivalent to the above definitions using the Cauchy–Schwarz inequality. Matrix norms induced by vector α- and β- norms
Suppose vector norms  and  are used for spaces  and  respectively, the corresponding operator norm is:In the special cases of  and , the induced matrix norms can be computed bywhere  is the i-th row of matrix . In the special cases of  and , the induced matrix norms can be computed bywhere  is the j-th column of matrix . Hence,  and  are the maximum row and column 2-norm of the matrix, respectively. Properties

Any operator norm is consistent  with the vector norms that induce it, giving

Suppose ; ; and  are operator norms induced by the respective pairs of vector norms ; ; and . Then,

this follows from

and

Square matrices
Suppose  is an operator norm on the space of square matrices 
induced by vector norms  and . Then, the operator norm is a sub-multiplicative matrix norm: 

Moreover, any such norm satisfies the inequality

for all positive integers r, where  is the spectral radius of . For symmetric or hermitian , we have equality in () for the 2-norm, since in this case the 2-norm is precisely the spectral radius of . For an arbitrary matrix, we may not have equality for any norm; a counterexample would be

which has vanishing spectral radius. In any case, for any matrix norm, we have the spectral radius formula:

Consistent and compatible norms
A matrix norm  on  is called consistent with a vector norm  on  and a vector norm  on , if:

for all  and all . In the special case of  and ,  is also called compatible with . All induced norms are consistent by definition. Also, any sub-multiplicative matrix norm on  induces a compatible vector norm on  by defining . "Entry-wise" matrix norms
These norms treat an  matrix as a vector of size , and use one of the familiar vector norms. For example, using the p-norm for vectors, , we get:

This is a different norm from the induced p-norm (see above) and the Schatten p-norm (see below), but the notation is the same. The special case p = 2 is the Frobenius norm, and p = ∞ yields the maximum norm. and  norms

Let  be the columns of matrix . From the original definition, the matrix  presents n data points in m-dimensional space. The  norm is the sum of the Euclidean norms of the columns of the matrix:

The  norm as an error function is more robust, since the error for each data point (a column) is not squared. It is used in robust data analysis and sparse coding. For , the  norm can be generalized to the  norm as follows:

Frobenius norm

When  for the  norm, it is called the Frobenius norm or the Hilbert–Schmidt norm, though the latter term is used more frequently in the context of operators on (possibly infinite-dimensional) Hilbert space. This norm can be defined in various ways:

where  are the singular values of . Recall that the trace function returns the sum of diagonal entries of a square matrix. The Frobenius norm is an extension of the Euclidean norm to  and comes from the Frobenius inner product on the space of all matrices. The Frobenius norm is sub-multiplicative and is very useful for numerical linear algebra. The sub-multiplicativity of Frobenius norm can be proved using Cauchy–Schwarz inequality. Frobenius norm is often easier to compute than induced norms, and has the useful property of being invariant under rotations (and unitary operations in general). That is,  for any unitary matrix . This property follows from the cyclic nature of the trace ():

and analogously:

where we have used the unitary nature of  (that is, ). It also satisfies

and 

where  is the Frobenius inner product, and Re is the real part of a complex number (irrelevant for real matrices)

Max norm

The max norm is the elementwise norm in the limit as  goes to infinity:

This norm is not sub-multiplicative. Note that in some literature (such as Communication complexity), an alternative definition of max-norm, also called the -norm, refers to the factorization norm:

Schatten norms

The Schatten p-norms arise when applying the p-norm to the vector of singular values of a matrix. If the singular values of the  matrix  are denoted by σi, then the Schatten p-norm is defined by

These norms again share the notation with the induced and entry-wise p-norms, but they are different. All Schatten norms are sub-multiplicative. They are also unitarily invariant, which means that  for all matrices  and all unitary matrices  and . The most familiar cases are p = 1, 2, ∞. The case p = 2 yields the Frobenius norm, introduced before. The case p = ∞ yields the spectral norm, which is the operator norm induced by the vector 2-norm (see above). Finally, p = 1 yields the nuclear norm (also known as the trace norm, or the Ky Fan 'n'-norm), defined as:

where  denotes a positive semidefinite matrix  such that . More precisely, since  is a positive semidefinite matrix, its square root is well-defined. The nuclear norm  is a convex envelope of the rank function , so it is often used in mathematical optimization to search for low-rank matrices. Combining von Neumann's trace inequality
with Hölder's inequality for Euclidean space
yields a version of Hölder's inequality
for Schatten norms
for
:

In particular, this implies the Schatten norm inequality

Monotone norms
A matrix norm  is called monotone if it is monotonic with respect to the Loewner order. Thus, a matrix norm is increasing if

The Frobenius norm and spectral norm are examples of monotone norms. Cut norms 
Another source of inspiration for matrix norms arises from considering a matrix as the adjacency matrix of a weighted, directed graph. The so-called "cut norm" measures how close the associated graph is to being bipartite:
 
where . Equivalent definitions (up to a constant factor) impose the conditions ; ; or . The cut-norm is equivalent to the induced operator norm , which is itself equivalent to another norm, called the Grothendieck norm. To define the Grothendieck norm, first note that a linear operator  is just a scalar, and thus extends to a linear operator on any . Moreover, given any choice of basis for  and , any linear operator  extends to a linear operator , by letting each matrix element on elements of  via scalar multiplication. The Grothendieck norm is the norm of that extended operator; in symbols:

The Grothendieck norm depends on choice of basis (usually taken to be the standard basis) and . Equivalence of norms

For any two matrix norms  and , we have that:

for some positive numbers r and s, for all matrices . In other words, all norms on  are equivalent; they induce the same topology on . This is true because the vector space  has the finite dimension . Moreover, for every vector norm  on , there exists a unique positive real number  such that  is a sub-multiplicative matrix norm for every . A sub-multiplicative matrix norm  is said to be minimal, if there exists no other sub-multiplicative matrix norm  satisfying . Examples of norm equivalence

Let  once again refer to the norm induced by the vector p-norm (as above in the Induced Norm section). For matrix  of rank , the following inequalities hold:

See also 

 Dual norm
 Logarithmic norm

Notes

References

Bibliography
 James W. Demmel, Applied Numerical Linear Algebra, section 1.7, published by SIAM, 1997. Carl D. Meyer, Matrix Analysis and Applied Linear Algebra, published by SIAM, 2000. John Watrous, Theory of Quantum Information, 2.3 Norms of operators, lecture notes, University of Waterloo, 2011. Kendall Atkinson, An Introduction to Numerical Analysis, published by John Wiley & Sons, Inc 1989

Norms (mathematics)
Linear algebra