itertools — Functions creating iterators for efficient looping¶New in version 2.3. This module implements a number of iterator building blocks inspired by constructs from the Haskell and SML programming languages. Each has been recast in a form suitable for Python. The module standardizes a core set of fast, memory efficient tools that are useful by themselves or in combination. Standardization helps avoid the readability and reliability problems which arise when many different individuals create their own slightly varying implementations, each with their own quirks and naming conventions. The tools are designed to combine readily with one another. This makes it easy to construct more specialized tools succinctly and efficiently in pure Python. For instance, SML provides a tabulation tool: tabulate(f) which produces a sequence f(0), f(1), .... This toolbox provides imap() and count() which can be combined to form imap(f, count()) and produce an equivalent result. Likewise, the functional tools are designed to work well with the high-speed functions provided by the operator module. Whether cast in pure python form or compiled code, tools that use iterators are more memory efficient (and often faster) than their list based counterparts. Adopting the principles of just-in-time manufacturing, they create data when and where needed instead of consuming memory with the computer equivalent of “inventory”. See also The Standard ML Basis Library, The Standard ML Basis Library. Haskell, A Purely Functional Language, Definition of Haskell and the Standard Libraries. Itertool functions¶The following module functions all construct and return iterators. Some provide streams of infinite length, so they should only be accessed by functions or loops that truncate the stream.
Examples¶The following examples show common uses for each tool and demonstrate ways they can be combined. # Show a dictionary sorted and grouped by value >>> from operator import itemgetter >>> d = dict(a=1, b=2, c=1, d=2, e=1, f=2, g=3) >>> di = sorted(d.iteritems(), key=itemgetter(1)) >>> for k, g in groupby(di, key=itemgetter(1)): ... print k, map(itemgetter(0), g) ... 1 ['a', 'c', 'e'] 2 ['b', 'd', 'f'] 3 ['g'] # Find runs of consecutive numbers using groupby. The key to the solution # is differencing with a range so that consecutive numbers all appear in # same group. >>> data = [ 1, 4,5,6, 10, 15,16,17,18, 22, 25,26,27,28] >>> for k, g in groupby(enumerate(data), lambda (i,x):i-x): ... print map(itemgetter(1), g) ... [1] [4, 5, 6] [10] [15, 16, 17, 18] [22] [25, 26, 27, 28] Recipes¶This section shows recipes for creating an extended toolset using the existing itertools as building blocks. The extended tools offer the same high performance as the underlying toolset. The superior memory performance is kept by processing elements one at a time rather than bringing the whole iterable into memory all at once. Code volume is kept small by linking the tools together in a functional style which helps eliminate temporary variables. High speed is retained by preferring “vectorized” building blocks over the use of for-loops and generators which incur interpreter overhead. def take(n, iterable):
"Return first n items of the iterable as a list"
return list(islice(iterable, n))
def enumerate(iterable, start=0):
return izip(count(start), iterable)
def tabulate(function, start=0):
"Return function(0), function(1), ..."
return imap(function, count(start))
def nth(iterable, n):
"Returns the nth item or empty list"
return list(islice(iterable, n, n+1))
def quantify(iterable, pred=bool):
"Count how many times the predicate is true"
return sum(imap(pred, iterable))
def padnone(iterable):
"""Returns the sequence elements and then returns None indefinitely.
Useful for emulating the behavior of the built-in map() function.
"""
return chain(iterable, repeat(None))
def ncycles(iterable, n):
"Returns the sequence elements n times"
return chain.from_iterable(repeat(iterable, n))
def dotproduct(vec1, vec2):
return sum(imap(operator.mul, vec1, vec2))
def flatten(listOfLists):
return list(chain.from_iterable(listOfLists))
def repeatfunc(func, times=None, *args):
"""Repeat calls to func with specified arguments.
Example: repeatfunc(random.random)
"""
if times is None:
return starmap(func, repeat(args))
return starmap(func, repeat(args, times))
def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = tee(iterable)
for elem in b:
break
return izip(a, b)
def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)
def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
# Recipe credited to George Sakkis
pending = len(iterables)
nexts = cycle(iter(it).next for it in iterables)
while pending:
try:
for next in nexts:
yield next()
except StopIteration:
pending -= 1
nexts = cycle(islice(nexts, pending))
def powerset(iterable):
"powerset('ab') --> set([]), set(['a']), set(['b']), set(['a', 'b'])"
# Recipe credited to Eric Raymond
pairs = [(2**i, x) for i, x in enumerate(iterable)]
for n in xrange(2**len(pairs)):
yield set(x for m, x in pairs if m&n)
def compress(data, selectors):
"compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F"
return (d for d, s in izip(data, selectors) if s)
def combinations_with_replacement(iterable, r):
"combinations_with_replacement('ABC', 3) --> AA AB AC BB BC CC"
pool = tuple(iterable)
n = len(pool)
indices = [0] * r
yield tuple(pool[i] for i in indices)
while 1:
for i in reversed(range(r)):
if indices[i] != n - 1:
break
else:
return
indices[i:] = [indices[i] + 1] * (r - i)
yield tuple(pool[i] for i in indices)
|