MIT OCW 18.06SC Sections – Lectures & Reading:
OCW Lecture 25: Symmetric Matrices & Positive Definitenesss
Strang 6.4 & 6.5
Symmetric Matrices
What is a Symmetric Matrix?
A matrix is symmetric if it is equal to its transpose. In Linear Algebra talk, we say
We can see symmetry in a matrix if its components are mirrored across its diagonal, such as
What is Special About Symmetric Matrices?
Symmetric matrices tie together a number of properties which make them simpler and easier to deal with. Those properties are:
- All eigenvalues are real
- Eigenvectors corresponding to different eigenvalues are orthogonal
- They are always diagonalizable
- They can be diagonalized using an orthogonal matrix
- Their pivots and eigenvalues are closely connected
Spectral Theorem
I don’t know how the Spectral Theorem was devised, nor what egghead deduction went into it. In a nutshell, it’s a diagonalization, in a simpler a la symmetric matrix kind of way, like this:
Q, as we’ve learned previously, represents an orthogonal matrix whose columns are composed of orthonormal eigenvectors. Big Lambda 𝚲 is a diagonal matrix of eigenvalues. Together, they make a much friendlier diagonalization than the standard
because
In other words, A is equal to matrix Q, its inverse, and a diagonal matrix of its eigenvalues. Once grokked (I’m still working on it), it means these matrices are simpler to work with. And anything that makes Linear Algebra simpler is great in my book. As Oscar Wilde said, “simple pleasures are the last refuge of the complex”
Real Eigenvalues and Orthogonal Eigenvectors
Even though many matrices can produce complex eigenvalues, symmetric matrices cannot. However, if a matrix is symmetric, all its eigenvalues are real – another win for simplicity, thank you.
Eigenvectors are orthogonal to each other in symmetric matrices. So
When i is not equal to j. Because of this, we can build an orthogonal matrix Q from normalized eigenvectors.
Complex Eigenvalues of Real Matrices
A real matrix can still have complex eigenvalues. Take, for example:
This matrix has eigenvalues of +/- i. While being a real matrix does NOT guarantee real eigenvalues, being symmetric DOES guarantee real eigenvalues. Ahh, simple pleasures…
The Connection between Pivots and Eigenvalues
Positive pivots often correspond to positive eigenvalues. Determinants equal the product of a matrix’s eigenvalues. The trace of a symmetric matrix is equal to the sum of its eigenvalues.
All Symmetric Matrices are Diagonalizable
Although not all matrices are diagonalizable, all symmetric matrices are.
The Big Picture
Symmetric matrices are important because they combine many major ideas in linear algebra into one well behaved matrix class. As covered, a symmetric matrix encapsulates
- elimination
- pivots
- determinants
- eigenvalues
- orthogonality
- diagonalization
As I understand it, a symmetric matrix is the ideal matrix in Linear Algebra.
Positive Definiteness
What is a Positive Definite Matrix?
A symmetric matrix is positive definite if
Take, for example, the following positive definite matrix:
and
Is positive unless x = 0.
Testing Pivots for Positive Definiteness
For symmetric matrices, a matrix is positive definite if and only if all its pivots are positive. This is nice because, you guessed it, it’s simpler. A n x n matrix’s eigenvalues are harder to compute than its pivots, especially as n increases. So, instead of finding the matrix’s eigenvalues, we can perform elimination and inspect the pivots. As we’ll see, that will also clue us in on the matrix’s eigenvalues.
Energy Based Definition
The quantity
is often interpreted as ‘energy’. Positive definite matrices guarantee that energy is always positive except at equilibrium (x = 0).
The “Five Equivalent Conditions”
For symmetric matrices, if one of the following properties is true, all of them are true, and the matrix is positive definite:
- All pivots are positive
- All eigenvalues are positive
- Upper left determinants are all positive
- The energy based definition holds true
- The symmetric matrix S is equal to matrix A and its transpose
The Big Picture
Positive definite matrices are ‘stable’ symmetric matrices. They connect:
- pivots
- eigenvalues
- determinants
- quadratic forms
- geometry
- Energy
into a unified idea.
Positive definite matrices always produce positive energy and behave in a stable, well-structured way.
Reflection:
This unit was very challenging for me. Taken in pieces, I understood the theory behind symmetric matrices and positive definite matrices. But making sense of it as a whole, blending all the pieces together, describing them with a panoply of symbols, is overwhelming. Moreover, the exercises were very abstract. Perhaps if I had some more computational exercises to drill these ideas, it would help. With that being said, it’s humbling to be so intellectually challenged. And it’s healthy to know where I need to make improvements.