Dave Beazley's mondo computer blog. [ homepage ]

Tuesday, February 02, 2010


A Context Manager for Timing Benchmarks

I spend a lot of time studying different aspects of Python, different implementation techniques, and so forth. As part of that, I often carry out little performance benchmarks. For small things, I'll often use the timeit module. For example:

>>> from timeit import timeit
>>> timeit("math.sin(2)","import math")
>>> timeit("sin(2)","from math import sin")

However, for larger blocks of code, I tend to just use the time module directly like this:

import time
start = time.time()
... some big calculation
end = time.time()
print("Whatever : %0.3f seconds" % (end-start))

Having typed that particular code template more often than I care to admit, it occurred to me that I really ought to just make a context manager for doing it. Something like this:

# benchmark.py
import time
class benchmark(object):
    def __init__(self,name):
        self.name = name
    def __enter__(self):
        self.start = time.time()
    def __exit__(self,ty,val,tb):
        end = time.time()
        print("%s : %0.3f seconds" % (self.name, end-self.start))
        return False

Now, I can just use that context manager whenever I want to do that kind of timing benchmark. For example:

# fileitertest.py
from benchmark import benchmark

with benchmark("iterate by lines (UTF-8)"):
     for line in open("biglog.txt",encoding='utf-8'):

with benchmark("iterate by lines (Latin-1)"):
     for line in open("biglog.txt",encoding='latin-1'):

with benchmark("iterate by lines (Binary)"):
     for line in open("biglog.txt","rb"):

If you run it, you might get output like this:

bash % python3 fileitertest.py
iterate by lines (UTF-8) : 3.903 seconds
iterate by lines (Latin-1) : 3.615 seconds
iterate by lines (Binary) : 1.886 seconds

Nice. I like it already!

Thanks! It's also a nice instructional use case for a homemade context manager!
I use a decorator for this:

def TimeFunction(func):
"""Decorator to time a function."""
def wrapper(*args, **kwargs):
t1 = time.time()
res = func(*args, **kwargs)
t2 = time.time()
print '%s took %0.3f ms' % (func.func_name, (t2-t1)*1000.0)
return res
return wrapper
I have a similar tool but I use pystones instead of seconds to make my measures comparables from one box to another.
@Tarek: that is interesting. I'm of course familiar with the overall pystones benchmark in the test suite, but not with how to run tests of targeted user code in pystone context. Do you have sample code I can see? Thanks.

@David: handy. I too write those time.time() calls a lot, and this should save me a few keystrokes.
Tarek, I'm also interested in this pystone conversion. Is there some description of that or a formula? I know of a general pystones benchmark, but don't know how it applies to more specific tests I might want to do myself.
My guess Tarek is referring to this:

I haven't tried it personally...
On Windows, time.clock() should be used instead of time.time().
Post a Comment

<< Home


08/01/2009 - 09/01/2009   09/01/2009 - 10/01/2009   10/01/2009 - 11/01/2009   11/01/2009 - 12/01/2009   12/01/2009 - 01/01/2010   01/01/2010 - 02/01/2010   02/01/2010 - 03/01/2010   04/01/2010 - 05/01/2010  

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]