Mercurial > pylearn

annotate pylearn/datasets/config.py @ 833:039e93a95c20

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
dataset.config uses logging for warnings
author James Bergstra <bergstrj@iro.umontreal.ca>
date Fri, 16 Oct 2009 12:14:11 -0400
parents f4729745bb58
children e7d1dd6a9785
rev   line source
504
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1 """Configuration options for datasets
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
3
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
4 Especially, the locations of data files.
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
5 """
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
6
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
7 import os, sys, logging
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
8 _logger = logging.getLogger('pylearn.datasets.config')
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
9 def debug(*msg): _logger.debug(' '.join(str(m) for m in msg))
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
10 def info(*msg): _logger.info(' '.join(str(m) for m in msg))
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
11 def warn(*msg): _logger.warn(' '.join(str(m) for m in msg))
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
12 def warning(*msg): _logger.warning(' '.join(str(m) for m in msg))
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
13 def error(*msg): _logger.error(' '.join(str(m) for m in msg))
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
14
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
15
655
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
16 def env_get(key, default, key2 = None):
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
17 if key2 and os.getenv(key) is None:
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
18 key=key2
653
d3d8f5a17909 print warning on undefined PYLEARN_DATA_ROOT
bergstra@mlp4.ais.sandbox
parents: 537
diff changeset
19 if os.getenv(key) is None:
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
20 if env_get.first_warning:
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
21 warning("Environment variable", key, 'is not set. Using default of', default)
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
22 env_get.first_warning = False
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
23 return default
818
f4729745bb58 backporting to 2.4
dumitru@deepnets.mtv.corp.google.com
parents: 655
diff changeset
24 else:
833
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
25 os.getenv(key)
039e93a95c20 dataset.config uses logging for warnings
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 818
diff changeset
26 env_get.first_warning = True
504
19ab9ce916e3 slightly more sophisticated system for finding the mnist data
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
27
505
74b3e65f5f24 added smallNorb dataset, switched to PYLEARN_DATA_ROOT
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 504
diff changeset
28 def data_root():
655
14d22ca1c8b5 if PYLEARN_DATA_ROOT don't exist try DBPATH.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 653
diff changeset
29 return env_get('PYLEARN_DATA_ROOT', os.getenv('HOME')+'/data', 'DBPATH')
505
74b3e65f5f24 added smallNorb dataset, switched to PYLEARN_DATA_ROOT
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 504
diff changeset
30