A fast randomized algorithm for overdetermined linear least-squares regression

We introduce a randomized algorithm for overdetermined linear least-squares regression. Given an arbitrary full-rank m x n matrix A with m >= n, any m x 1 vector b, and any positive real number 6, the procedure computes an n x 1 vector x such that x minimizes the Euclidean norm parallel to Ax - b parallel to to relative precision epsilon. The algorithm typically requires O((log(n) + log(1/epsilon))mn + n(3)) floating-point operations. This cost is less than the O(mn(2)) required by the classical schemes based on QR-decompositions or bidiagonalization. We present several numerical examples illustrating the performance of the algorithm.