Anscombe’s quartet is a set of four toy datasets that look very different, but many of their statistics coincide. They were developed by Francis Anscombe as a striking visual to show that even for small datasets, blindly examining their statistical properties without considering their structure can mislead.

Particularly, the four datasets have the same least squares regression line. While the second dataset is a clear example of a nonlinear correlation which cannot be accurately captured by any linear model, the third dataset is actually perfectly linear, with no noise, but just a single outlier that shifts the regression line considerably.

Support vector regression is an extension of the support vector
machine idea to tackle the regression problem. It is based on the
observation that a SVM classifier builds its decision boundary as a
function of a (small) subset of training points. For regression, SVR
fits a *tube* that is robust to noise within a width
[latex]\epsilon[/latex]. For this particular example, using a small
width makes the solution robust to the obvious outlier. For very small
but non-zero [latex]\epsilon[/latex], the solution is a combination of
the outlier and on two other points. For [latex]\epsilon=0[/latex], you
can see that every point except a non-outlier is highlighted. This is
actually the perfect solution but very dense.

[][]

Every frame displays the global mean squared error and the true mean
squared error, *i.e.* over the inlying points. If the epsilon size is
well chosen, SVR can perform robustly with a sparse solution. Since our
interest was in avoiding the outlier, we assumed no noise in the inlying
data, so a very small epsilon is perfect. For real data a larger epsilon
is often useful because of variability in the data. When adding noise,
SVR still manages to avoid the outlier, but when the tube width becomes
zero, the solution is again very dense, very non-parametric.

[][]

Here is the code you can use to play around with this.

[gist id=2815589]

[]: http://localhost:8001/wp-content/uploads/2012/05/svr.gif

[]: http://localhost:8001/wp-content/uploads/2012/05/svr_noise.gif

## Comments !