Mercurial > pylearn

annotate pylearn/dataset_ops/memo.py @ 998:8ba8b08e0442

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added the image_patches dataset used in RanzatoHinton2010 modified mcRBM to use it.
author James Bergstra <bergstrj@iro.umontreal.ca>
date Tue, 24 Aug 2010 16:51:53 -0400
parents 5c7374bd127c
children c9ec065ff736
rev   line source
832
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1 """Provide a decorator that caches expensive functions
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2 """
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
3 import logging
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
4 _logger = logging.getLogger(__file__)
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
5 info = _logger.info
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
6 def infop(*args):
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
7 info(' '.join(str(a) for a in args))
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
8
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
9 def memo(f):
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
10 #TODO: support kwargs to rval. This requires looking up the names of f's parameters to
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
11 # construct a proper key.
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
12
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
13 #TODO: use weak references instead of a normal dict so that the cache doesn't prevent
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
14 # garbage collection
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
15 cache = {}
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
16 def rval(*args):
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
17 if args not in cache:
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
18 cache[args] = f(*args)
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
19 return cache[args]
840
7ccce98da2b6 added forget function to memo in dataset_ops.memo
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 832
diff changeset
20 def forget():
7ccce98da2b6 added forget function to memo in dataset_ops.memo
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 832
diff changeset
21 for k in cache.keys():
7ccce98da2b6 added forget function to memo in dataset_ops.memo
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 832
diff changeset
22 del cache[k]
866
9b13f7bb00ef added cache member to memo-ized functions
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 840
diff changeset
23 rval.cache = cache
840
7ccce98da2b6 added forget function to memo in dataset_ops.memo
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 832
diff changeset
24 rval.forget = forget
832
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
25 rval.__name__ = 'memo@%s'%f.__name__
858
f11881a265ee export 'cache' variable
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 840
diff changeset
26 rval.cache = cache
832
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
27 return rval
67b92a42f86b added dataset_ops
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
28