The scikit-learn-speed project now has memory usage benchmarking!
This was accomplished by building on what I described in my recent
posts, specifically the extensions to Fabian’s [memory_profiler][] that
you can find in my fork, but they will be merged upstream soon. The
key element is the %magic_memit function whose development I blogged
about on several occasions. I plugged this into vbench
in a similar way to how the timings are computed, all with great success.
Here is a screenshot of the way a simple benchmark looks now, with just a few data points.
[caption id=”attachment_464” align=”aligncenter” width=”600”][![A screenshot showing generated output from the scikit-learn-speed project, illustrating memory usage benchmarking.][]][] Memory benchmarking in scikit-learn-speed powered by vbench.[/caption]
You can check it out and use it yourself for your benchmarks, but you need to use the vbench from the memory branch on my fork.
Of course, there are some important caveats. I am running this on my
laptop, which runs OS X Lion, so, under the effect of this
bug, I hardcoded the ‘-i‘ so the memory benchmarks are not
realistic. Also, the y-range should probably be forced wider, because
the plots look erratic, showing the very small noise at a large-scale.
[![A screenshot showing generated output from the scikit-learn-speed project, illustrating memory usage benchmarking.][]]: http://localhost:8001/wp-content/uploads/2012/07/vbench1.png
Comments !