Some linear algebra application ideas – Michigan Math In Action

Last Fall, I took over our “Applied Linear Algebra” course. This course is targeted at an audience majoring primarily in either electrical engineering and computer science or else in industrial and operations engineering. I’m not an applied mathematician myself, but I really wanted to get nontrivial applications of Linear Algebra into the curriculum.

We did this in two ways — first, with large multi-week group projects, and second with classroom demos using real data and MATLAB or Mathematica scripts. For the first projects, we chose the applications and walked them through the computation. For the second, we challenged them to go out into the world and find us a real application, which they presented in a packed poster fair on the last day of class.

Afterwards, some other instructors told me they were surprised at how many examples I found. So I wrote up a quick note with my favorite examples. Here is a cleaned up version of that note. It mixes together in class demos and group projects, as I think most of these could work at either length.

One thing I’d like to do better in future terms is see if I can find a way to have the students do in-class experimentation with some of these ideas. When I brought this material into class, I was using my laptop and premade scripts on the projector, taking questions and ideas for what sort of computations to run, but still fundamentally controlling the presentation. The big challenge is that, although most of the students have used MATLAB, and many are CS majors, most of them are not actually fluent coders who can think of an idea and immediately make a MATLAB script to do it. (The small challenge is getting enough laptops and desk space, but I believe I could solve this one.) Still, it would be great if we could.

One deficiency of this list is that I don’t have a lot of good applications for the first third of the term — solving linear equations, reduced row echelon form, subspaces, bases, dimension. I hope I’ll solve this in future terms.

Orthogonality

Discrete Fourier analysis I pointed out that the matrix of regularly sampled sines and cosines is orthogonal (with appropriate weighting). I justified this heuristically, pointing out that the orthogonality was easy for the integral version. I projected 10 years of Detroit weather data onto its leading Fourier terms.

Least squares I fit MSU tuition data to a quadratic. (Remember to correct for inflation! ) By the way, I would like to scowl at UMich for not making this data comparably easy to find.

Least squares models are omnipresent in statistics, so you can choose any example you like. I wanted to stay away from best fit lines, though, because I wanted to make the point that you can use least squares with a model of the form $y = a x^2 +b x + c$ ; the nonlinearity in $x$ is not a problem. I
think most students take a while to understand this, but it is good to make them.

This example was a short one, so I also used it to try to convince students that they had the tools to get at real datasets and work with them. Of course, the better way to do this would be to make THEM seek out a real data set…

Eigenvalues

Eigenvalues and oscillations of mechanical systems Take three masses of mass $m$ on a line, with masses 1 and 2, and masses 2 and 3, joined by a spring of spring constant $k$ . We get a system of ODE’s

$\begin{array}{rcl} m x''_1 &=& k(x_2-x_1) \\ m x''_2 &=& k(x_1-x_2) + k(x_2-x_3) \\ m x''_3 &=& k(x_3-x_2) \\ \end{array}$

Guessing a solution $x_j = a_j \cos (\omega t)$ , you find that $\omega$ is an eigenvalue and $(a_1, a_2, a_3)$ is the eigenvector. After I worked this small example, I talked about oscillation of molecules and of large mechanical structures (via the finite element method). I returned to this example in when we talked about quadratic forms, where I could point out that the matrix here is symmetric, and the corresponding (positive semidefinite!) quadratic form is the energy stored in the springs.

Demography and dynamical systems I took the data from page 39 of the Pew report and investigated how the distribution of religious demographic groups in the US would change over time in terms of eigenvectors. This was basically replicating 538’s analysis, but was (I hope!) more interesting than talking about rabbits and foxes.

Thinking about these issues also gave me a new way of thinking about society. We tend to think about different segments in society as vying for complete dominance — will we be liberal or conservative? religious or secular? Any division seems a temporary state, simply waiting for one side to dominate. But, if your matrix has off diagonal terms, then the equilibrium state will actually be a mix of populations. You don’t need math to see this, but I feel like I only got an intuition for it by seeing the math.

Random walks I took a couple of street networks and showed how a random walk in them would spread out in terms of eigenvalues. I showed how the eigenvalue gap controlled the connectivity of the graph.

Quadratic forms

Spectral graph partitioning David Gleich has a great intro to this method for finding clusters in a graph of connections. I started with some artificial examples and then finished with this dataset of links between several hundred blogs. (That data is a decade old now! I’ll need to get fresher data next time.)

I also did some examples of spectral graph drawing.

There are tons of great examples of graphs available for download; check out these links 1, 2, 3, 4.

Principal component analysis of statistical data I used Cosma Shalizi’s file of 17 features of 390 cars and extracted principal components from it.

Again, it would be great to make the students do something like this instead!

SVD for image compression I took a black and white bitmap and showed them how different numbers of singular values made the image come in. I’ve been told that this is a fake example, and no one actually does image compression this way. That’s too bad, because it is beautiful watching it work.

Optimal portfolio allocation I didn’t do it this term, but other times I’ve taught about optimal portfolio allocation. The problem is the following: You can distribute your money among various investments whose results are normally distributed, and correlated with each other. You want to choose a vector $\vec{r}$ which maximizes expected return while taking a fixed level of risk. The problem is to maximize a linear functional $\vec{a}^T \vec{r}$ with respect to a quadratic constraint $\vec{r}^T Q \vec{r} = \mbox{constant}$ . The answer is that you should pick $\vec{r}$ proportional to $Q^{-1} \vec{a}$ , and it is a nice example of the power of changing coordinates.