A while back, Bob L. Sturm blogged about a similar implementation of
OMP to the one in scikit-learn. Instead of using the Cholesky
decomposition like we did, his Matlab code uses the QR decomposition, to
a similar (or maybe even identical) outcome, in theory. So lucky that
Alejandro pointed out …

After intense code optimization work, my implementation of OMP finally
beat least-angle regression! This was the primary issue discussed during
the pull request, so once performance was taken care of, the code was
ready for merge. Orthogonal matching pursuit is now available in
scikits.learn as a sparse linear regression …