Mercurial > pylearn

annotate pylearn/datasets/config.py @ 1285:976539956475

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
adding tinyimages
author James Bergstra <bergstrj@iro.umontreal.ca>
date Thu, 23 Sep 2010 19:12:08 -0400
parents e7d1dd6a9785
children 58dff11840f0
rev   line source
504
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1 """Configuration options for datasets
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
3
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
4 Especially, the locations of data files.
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
5 """
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
6
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
7 import os, sys, logging
1285
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
8 def _logger(): return logging.getLogger('pylearn.datasets.config')
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
9 def debug(*msg): _logger().debug(' '.join(str(m) for m in msg))
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
10 def info(*msg): _logger().info(' '.join(str(m) for m in msg))
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
11 def warn(*msg): _logger().warn(' '.join(str(m) for m in msg))
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
12 def warning(*msg): _logger().warning(' '.join(str(m) for m in msg))
976539956475 adding tinyimages
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 848
diff changeset
13 def error(*msg): _logger().error(' '.join(str(m) for m in msg))
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
14
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
15
655
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
16 def env_get(key, default, key2 = None):
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
17 if key2 and os.getenv(key) is None:
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
18 key=key2
653
d3d8f5a17909 print warning on undefined PYLEARN_DATA_ROOT
bergstra@mlp4.ais.sandbox
parents: 537
diff changeset
19 if os.getenv(key) is None:
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
20 if env_get.first_warning:
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
21 warning("Environment variable", key, 'is not set. Using default of', default)
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
22 env_get.first_warning = False
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
23 return default
818
f4729745bb58 backporting to 2.4
dumitru@deepnets.mtv.corp.google.com
parents: 655
diff changeset
24 else:
848
e7d1dd6a9785 Fix missing "return"
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 833
diff changeset
25 return os.getenv(key)
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
26 env_get.first_warning = True
504
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
27
505
74b3e65f5f24 added smallNorb dataset, switched to PYLEARN_DATA_ROOT
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 504
diff changeset
28 def data_root():
655
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
29 return env_get('PYLEARN_DATA_ROOT', os.getenv('HOME')+'/data', 'DBPATH')
505
74b3e65f5f24 added smallNorb dataset, switched to PYLEARN_DATA_ROOT
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 504
diff changeset
30