Visualization of the singular value decomposition

The singular value decomposition of a square (real, complex, or quaternionic) matrix A is a factorization A=UDV where U,V are unitary matrices and D is a diagonal matrix with positive non-increasing entries. The diagonal entries of the singular value decomposition are known as the singular values of A, and these singular values are unique. Whenever the diagonal entries are distinct (this happens almost everywhere), the unitary matrices U,V are unique as well.
On the left side of this visualization, we show the real matrix A where A changes over time, and on the right side, we show the orthogonal matrix U in the singular value decomposition U.
In the visualization, the matrix A begins as a random real matrix with independent standard Gaussian entries, but for the second half of the visualization, the matrix A gradually becomes more symmetric.
While the singular value decomposition is always unique, this decomposition is unstable
The gradients of the eigenvalues and eigenvectors of a matrix were computed in the 1985 paper ON DIFFERENTIATING EIGENVALUES AND EIGENVECTORS by Jan Magnus. We can apply those calculations to produce expressions for the gradients of the singular vectors of matrix.
We observe that in this visualization, much of the instability of the singular value decomposition is a result of two singular values swapping their ordering, but this is not the only cause of instability. In a future visualization, I will probably show how much the swapping of the ordering of singular values contributes to the instability of the singular value decomposition.
The notion of a singular value decomposition is not my own. I am making this visualization to demonstrate a potential stability issue with the singular value decomposition. In addition to stability issues, one also encounters difficulty in generalizing the notion of a singular value decomposition to higher order objects such as tensors, collections of square matrices, and quantum channels/completely positive superoperators.
It is known that if A(t) is an analytic Hermitian valued matrix, then there are analytic vector valued functions r_1,...,r_n and analytic functions s_1,...,s_n where (r_1(t),...,r_n(t)) is the set of all eigenvectors of A(t) and (s_1(t),...,s_n(t)) are their corresponding eigenvalues counting multiplicity, and r_1,...,r_n,s_1,...,s_n are essentially unique by analytic continuation. In other words, if the matrix A varies smoothly enough, then we can make its singular vectors also vary smoothly.
Unless otherwise stated, all algorithms featured on this channel are my own. You can go to github.com/spo... to support my research on machine learning algorithms. I am also available to consult on the use of safe and interpretable AI for your business. I am designing machine learning algorithms for AI safety such as LSRDRs. In particular, my algorithms are designed to be more predictable and understandable to humans than other machine learning algorithms, and my algorithms can be used to interpret more complex AI systems such as neural networks. With more understandable AI, we can ensure that AI systems will be used responsibly and that we will avoid catastrophic AI scenarios. There is currently nobody else who is working on LSRDRs, so your support will ensure a unique approach to AI safety.

[@JustMeJustADude]

Are there any visual patterns that you yourself find interesting or remarkable?

[@josephvanname3377]

On the left side, the entries vary independently of each other, so the is nothing unremarkable when we just look at the left matrix. But the right side of the matrix is more remarkable especially when compared to the left side. 1. It is quite obvious that the right matrix evolves at a faster rate than the left matrix. 2. Much (but not all) of the chaos in the right matrix is due to column swaps. I will make another visualization without the row swaps (I need to generalize the singular value decomposition so that the entries on the diagonal entry are unordered to do this) so that we can see what that is like (making the visualization without column swaps will also give people a better sense of the stability of the singular value decomposition). 3. The amount of chaos in each of the columns varies with time. If we take a particular column, then that column is sometimes inactive, but at other times that column is highly active. This indicates that the instability stems from the interactions that singular vectors have with each other. 4. Columns tend to interact with their neighbors. We do not see columns that are far from each other interacting with each other. This means that if vectors have nearby singular values, then those vectors will blend in with each other and mix. These patterns become more apparent as one becomes familiar with the singular value decomposition and related results such as the polar decomposition and spectral theorem for normal or at least Hermitian operators. But the matrices on the left side are random matrices and each frame the matrix on the right side is a random orthogonal matrix, so since we are dealing with random matrices that move in random directions (with some momentum for smoothness), one should not look too deeply for patterns. Added later: It turns out that except for the column swaps, the orthogonal vectors vary smoothly over time. This visualization does not show this phenomenon very well, but I will make an upcoming visualization which clearly shows this phenomenon.