Memory benchmarking with vbench

The scikit-learn-speed project now has memory usage benchmarking!

This was accomplished by building on what I described in my recent posts, specifically the extensions to Fabian’s [memory_profiler][] that you can find in my fork, but they will be merged upstream soon. The key element is the %magic_memit function whose development I blogged about on several occasions. I plugged this into vbench in a similar way to how the timings are computed, all with great success.

Here is a screenshot of the way a simple benchmark looks now, with just a few data points.

[caption id=”attachment_464” align=”aligncenter” width=”600”][![A screenshot showing generated output from the scikit-learn-speed project, illustrating memory usage benchmarking.][]][] Memory benchmarking in scikit-learn-speed powered by vbench.[/caption]

You can check it out and use it yourself for your benchmarks, but you need to use the vbench from the memory branch on my fork.

Of course, there are some important caveats. I am running this on my laptop, which runs OS X Lion, so, under the effect of this bug, I hardcoded the ‘-i so the memory benchmarks are not realistic. Also, the y-range should probably be forced wider, because the plots look erratic, showing the very small noise at a large-scale.

[A screenshot showing generated output from the scikit-learn-speed project, illustrating memory usage benchmarking.]: http://localhost:8001/wp-content/uploads/2012/07/vbench1.png “Memory benchmarking in scikit-learn-speed powered by vbench.” [![A screenshot showing generated output from the scikit-learn-speed project, illustrating memory usage benchmarking.][]]: http://localhost:8001/wp-content/uploads/2012/07/vbench1.png

Comments !