Mercurial > ift6266

annotate deep/crbm/mnist_config.py.example @ 339:ffbf0e41bcee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
author fsavard
date Sat, 17 Apr 2010 20:29:18 -0400
parents
children 523e7b87c521
rev   line source
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
1 # ----------------------------------------------------------------------------
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
2 # BEGIN EXPERIMENT ISOLATION CODE
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
3
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
4 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
5 This makes sure we use the codebase clone created for this experiment.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
6 I.e. if you want to make modifications to the codebase but don't want your
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
7 running experiment code to be impacted by those changes, first copy the
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
8 codebase somewhere, and configure this section. It will make sure we import
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
9 from the right place.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
10
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
11 MUST BE DONE BEFORE IMPORTING ANYTHING ELSE
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
12 (Leave this comment there so others will understand what's going on)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
13 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
14
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
15 # Place where you copied modules that should be frozen for this experiment
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
16 codebase_clone_path = "/u/savardf/ift6266/experiment_clones/ift6266_mnistcrbm_exp1"
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
17
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
18 # Places where there might be conflicting modules from your $PYTHONPATH
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
19 remove_these_from_pythonpath = ["/u/savardf/ift6266/dev_code"]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
20
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
21 import sys
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
22 sys.path[0:0] = [codebase_clone_path]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
23
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
24 # remove paths we specifically don't want in $PYTHONPATH
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
25 for bad_path in remove_these_from_pythonpath:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
26 sys.path[:] = [el for el in sys.path if not el in (bad_path, bad_path+"/")]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
27
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
28 # Make the imports
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
29 import ift6266
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
30
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
31 # Just making sure we're importing from the right place
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
32 modules_to_check = [ift6266]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
33 for module in modules_to_check:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
34 if not codebase_clone_path in module.__path__[0]:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
35 raise RuntimeError("Module loaded from incorrect path "+module.__path__[0])
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
36
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
37 # Path to pass to jobman sqlschedule. IMPORTANT TO CHANGE TO REFLECT YOUR CLONE.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
38 # Make sure this is accessible from the default $PYTHONPATH (in your .bashrc)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
39 # (and make sure every subdirectory has its __init__.py file)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
40 EXPERIMENT_PATH = "ift6266_mnistcrbm_exp1.ift6266.deep.crbm.mnist_crbm.jobman_entrypoint"
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
41
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
42 # END EXPERIMENT ISOLATION CODE
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
43 # ----------------------------------------------------------------------------
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
44
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
45 from jobman import DD
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
46
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
47 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
48 These are parameters used by mnist_crbm.py. They'll end up as globals in there.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
49
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
50 Rename this file to config.py and configure as needed.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
51 DON'T add the renamed file to the repository, as others might use it
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
52 without realizing it, with dire consequences.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
53 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
54
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
55 # change "sandbox" when you're ready
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
56 JOBDB = 'postgres://ift6266h10@gershwin/ift6266h10_db/fsavard_mnistcrbm_exp1'
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
57
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
58 # Set this to True when you want to run cluster tests, ie. you want
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
59 # to run on the cluster, many jobs, but want to reduce the training
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
60 # set size and the number of epochs, so you know everything runs
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
61 # fine on the cluster.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
62 # Set this PRIOR to inserting your test jobs in the DB.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
63 TEST_CONFIG = False
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
64
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
65 # save params at training end
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
66 SAVE_PARAMS = True
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
67
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
68 IMAGE_OUTPUT_DIR = 'img/'
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
69
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
70 # number of minibatches before taking means for valid error etc.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
71 REDUCE_EVERY = 100
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
72
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
73 # print series to stdout too (otherwise just produce the HDF5 file)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
74 SERIES_STDOUT_TOO = False
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
75
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
76 VISUALIZE_EVERY = 20000
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
77 GIBBS_STEPS_IN_VIZ_CHAIN = 1000
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
78
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
79 # This is to configure insertion of jobs on the cluster.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
80 # Possible values the hyperparameters can take. These are then
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
81 # combined with produit_cartesien_jobs so we get a list of all
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
82 # possible combinations, each one resulting in a job inserted
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
83 # in the jobman DB.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
84 JOB_VALS = {'learning_rate': [1.0, 0.1, 0.01],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
85 'sparsity_lambda': [3.0,0.5],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
86 'sparsity_p': [0.3,0.05],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
87 'num_filters': [40,15],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
88 'filter_size': [12,7],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
89 'minibatch_size': [20]}
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
90
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
91 # Just useful for tests... minimal number of epochs
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
92 # Useful when launching a single local job
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
93 DEFAULT_STATE = DD({'learning_rate': 0.1,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
94 'sparsity_lambda': 1.0,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
95 'sparsity_p': 0.05,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
96 'num_filters': 40,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
97 'filter_size': 12,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
98 'minibatch_size': 10})
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
99
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
100 # To reinsert duplicate of jobs that crashed
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
101 REINSERT_COLS = ['learning_rate','sparsity_lambda','sparsity_p','num_filters','filter_size','minibatch_size','dupe']
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
102 #REINSERT_JOB_VALS = [\
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
103 # [,2],]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
104