Mercurial > ift6266

annotate deep/crbm/mnist_config.py.example @ 340:523e7b87c521

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Corrected a few bugs, no new features. Supposedly ready to run on cluster.
author fsavard
date Sun, 18 Apr 2010 11:39:24 -0400
parents ffbf0e41bcee
children 74287f0371bf
rev   line source
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
1 # ----------------------------------------------------------------------------
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
2 # BEGIN EXPERIMENT ISOLATION CODE
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
3
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
4 def isolate_experiment():
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
5 '''
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
6 This makes sure we use the codebase clone created for this experiment.
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
7 I.e. if you want to make modifications to the codebase but don't want your
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
8 running experiment code to be impacted by those changes, first copy the
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
9 codebase somewhere, and configure this section. It will make sure we import
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
10 from the right place.
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
11
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
12 MUST BE DONE BEFORE IMPORTING ANYTHING ELSE
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
13 (Leave this comment there so others will understand what's going on)
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
14 '''
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
15
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
16 # Place where you copied modules that should be frozen for this experiment
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
17 codebase_clone_path = "/u/savardf/ift6266/experiment_clones/ift6266_mnistcrbm_exp1"
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
18
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
19 # Places where there might be conflicting modules from your $PYTHONPATH
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
20 remove_these_from_pythonpath = ["/u/savardf/ift6266/dev_code"]
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
21
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
22 import sys
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
23 sys.path[0:0] = [codebase_clone_path]
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
24
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
25 # remove paths we specifically don't want in $PYTHONPATH
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
26 for bad_path in remove_these_from_pythonpath:
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
27 sys.path[:] = [el for el in sys.path if not el in (bad_path, bad_path+"/")]
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
28
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
29 # Make the imports
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
30 import ift6266
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
31
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
32 # Just making sure we're importing from the right place
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
33 modules_to_check = [ift6266]
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
34 for module in modules_to_check:
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
35 if not codebase_clone_path in module.__path__[0]:
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
36 raise RuntimeError("Module loaded from incorrect path "+module.__path__[0])
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
37
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
38 # Path to pass to jobman sqlschedule. IMPORTANT TO CHANGE TO REFLECT YOUR CLONE.
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
39 # Make sure this is accessible from the default $PYTHONPATH (in your .bashrc)
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
40 # (and make sure every subdirectory has its __init__.py file)
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
41 EXPERIMENT_PATH = "ift6266_mnistcrbm_exp1.ift6266.deep.crbm.mnist_crbm.jobman_entrypoint"
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
42
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
43 # END EXPERIMENT ISOLATION CODE
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
44 # ----------------------------------------------------------------------------
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
45
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
46 from jobman import DD
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
47
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
48 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
49 These are parameters used by mnist_crbm.py. They'll end up as globals in there.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
50
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
51 Rename this file to config.py and configure as needed.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
52 DON'T add the renamed file to the repository, as others might use it
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
53 without realizing it, with dire consequences.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
54 '''
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
55
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
56 # change "sandbox" when you're ready
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
57 JOBDB = 'postgres://ift6266h10@gershwin/ift6266h10_sandbox_db/yourtablenamehere'
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
58
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
59 # Set this to True when you want to run cluster tests, ie. you want
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
60 # to run on the cluster, many jobs, but want to reduce the training
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
61 # set size and the number of epochs, so you know everything runs
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
62 # fine on the cluster.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
63 # Set this PRIOR to inserting your test jobs in the DB.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
64 TEST_CONFIG = False
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
65
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
66 # save params at training end
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
67 SAVE_PARAMS = True
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
68
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
69 IMAGE_OUTPUT_DIR = 'img/'
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
70
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
71 # number of minibatches before taking means for valid error etc.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
72 REDUCE_EVERY = 100
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
73
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
74 # print series to stdout too (otherwise just produce the HDF5 file)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
75 SERIES_STDOUT_TOO = False
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
76
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
77 VISUALIZE_EVERY = 20000
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
78 GIBBS_STEPS_IN_VIZ_CHAIN = 1000
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
79
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
80 if TEST_CONFIG:
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
81 REDUCE_EVERY = 10
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
82 VISUALIZE_EVERY = 20
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
83
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
84 # This is to configure insertion of jobs on the cluster.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
85 # Possible values the hyperparameters can take. These are then
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
86 # combined with produit_cartesien_jobs so we get a list of all
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
87 # possible combinations, each one resulting in a job inserted
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
88 # in the jobman DB.
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
89 JOB_VALS = {'learning_rate': [1.0, 0.1, 0.01],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
90 'sparsity_lambda': [3.0,0.5],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
91 'sparsity_p': [0.3,0.05],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
92 'num_filters': [40,15],
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
93 'filter_size': [12,7],
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
94 'minibatch_size': [20],
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
95 'num_epochs': [20]}
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
96
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
97 # Just useful for tests... minimal number of epochs
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
98 # Useful when launching a single local job
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
99 DEFAULT_STATE = DD({'learning_rate': 0.1,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
100 'sparsity_lambda': 1.0,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
101 'sparsity_p': 0.05,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
102 'num_filters': 40,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
103 'filter_size': 12,
340
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
104 'minibatch_size': 10,
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster.
fsavard
parents: 339
diff changeset
105 'num_epochs': 20})
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
106
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
107 # To reinsert duplicate of jobs that crashed
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
108 REINSERT_COLS = ['learning_rate','sparsity_lambda','sparsity_p','num_filters','filter_size','minibatch_size','dupe']
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
109 #REINSERT_JOB_VALS = [\
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
110 # [,2],]
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents:
diff changeset
111