SVD: Selected References

Thresholded Correlation Matrix more easily computed with SVD

“A Tutorial on Principal Component Analysis”, Jonathon Shlens

Mathematical Facts about SVD (with view to Signal Processing) by S. J. Orfanidis

“Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions”, Nathan Halko, PerGunnar Martinsson, Joel A. Tropp. A nice approach showing how to scale SVD to really large datasets.

Blog on benchmarking sparse vs. dense SVD in scipy (Dense SVD is faster but takes more memory and is not applicable to large sparse matrices)

Computation of the Singular Value Decomposition using MeshConnected Processors by Brent, Luk, Loan (an old paper showing a number of algorithms for SVD)

Versatality of Singular Value Decomposition by Ravi Kannan ( backup link)

Topic Modeling: A Provable Spectral Method by Ravi Kannan (Topic Modeling: A Provable Algorithm)

Clustering: Does theory help? by Ravi Kannan (backup link to slides)

Theory and Big Data by Ravi Kannan (how to apply provably fit a mixture of k Gaussians with the SVD)

Low Rank Approximation and Regression in Input Sparsity Time by Kenneth L. Clarkson and David P. Woodruff (Nearly best rank k approx to A can be found in time linear in the number of nonzero entries in A)

Simple and Deterministic Matrix Sketching by Edo Liberty (essentially running an algorithm for online SVD with small memory; very simple to implement but requires computing the SVD on a small matrix a large number of times.)