Mercurial > ift6266

annotate deep/deep_mlp/job.py @ 644:e63d23c7c9fb

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
reviews aistats finales
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Thu, 24 Mar 2011 17:05:05 -0400
parents 75dbbe409578
children
rev   line source
626
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
1 #!/usr/bin/env python
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
2 # coding: utf-8
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
3
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
4 '''
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
5 Launching
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
6
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
7 jobman sqlschedules postgres://ift6266h10@gershwin/ift6266h10_sandbox_db/mlp_dumi mlp_jobman.experiment mlp_jobman.conf
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
8 'n_hidden={{500,1000,2000}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
9 'n_hidden_layers={{2,3}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
10 'train_on={{NIST,NISTP,P07}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
11 'train_subset={{DIGITS_ONLY,ALL}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
12 'learning_rate_log10={{-1.,-2.,-3.}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
13
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
14 in mlp_jobman.conf:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
15 rng_seed=1234
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
16 L1_reg=0.0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
17 L2_reg=0.0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
18 n_epochs=10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
19 minibatch_size=20
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
20 '''
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
21
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
22 import os, sys, copy, operator, time
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
23 import theano
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
24 import theano.tensor as T
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
25 import numpy
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
26 from mlp import MLP
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
27 from ift6266 import datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
28 from pylearn.io.seriestables import *
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
29 import tables
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
30 from jobman.tools import DD
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
31
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
32 N_INPUTS = 32*32
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
33 REDUCE_EVERY = 250
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
34
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
35 TEST_RUN = False
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
36
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
37 TEST_HP = DD({'n_hidden':200,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
38 'n_hidden_layers': 2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
39 'train_on':'NIST',
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
40 'train_subset':'ALL',
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
41 'learning_rate_log10':-2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
42 'rng_seed':1234,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
43 'L1_reg':0.0,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
44 'L2_reg':0.0,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
45 'n_epochs':2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
46 'minibatch_size':20})
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
47
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
48 ###########################################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
49 # digits datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
50 # nist_digits is already in NIST_PATH and in ift6266.datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
51 # NOTE: for these datasets the test and valid sets are wrong
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
52 # (don't correspond to the training set... they're just placeholders)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
53
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
54 from ift6266.datasets.defs import NIST_PATH, DATA_PATH
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
55 TRANSFORMED_DIGITS_PATH = '/data/lisatmp/ift6266h10/data/transformed_digits'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
56
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
57 P07_digits = FTDataSet(\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
58 train_data = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
59 'data/P07_train'+str(i)+'_data.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
60 for i in range(0, 100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
61 train_lbl = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
62 'data/P07_train'+str(i)+'_labels.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
63 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
64 test_data = [os.path.join(DATA_PATH,'data/P07_test_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
65 test_lbl = [os.path.join(DATA_PATH,'data/P07_test_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
66 valid_data = [os.path.join(DATA_PATH,'data/P07_valid_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
67 valid_lbl = [os.path.join(DATA_PATH,'data/P07_valid_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
68 indtype=theano.config.floatX, inscale=255., maxsize=None)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
69
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
70 #Added PNIST
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
71 PNIST07_digits = FTDataSet(train_data = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
72 'PNIST07_train'+str(i)+'_data.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
73 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
74 train_lbl = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
75 'PNIST07_train'+str(i)+'_labels.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
76 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
77 test_data = [os.path.join(DATA_PATH,'data/PNIST07_test_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
78 test_lbl = [os.path.join(DATA_PATH,'data/PNIST07_test_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
79 valid_data = [os.path.join(DATA_PATH,'data/PNIST07_valid_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
80 valid_lbl = [os.path.join(DATA_PATH,'data/PNIST07_valid_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
81 indtype=theano.config.floatX, inscale=255., maxsize=None)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
82
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
83
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
84 # building valid_test_datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
85 # - on veut des dataset_obj pour les 3 datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
86 # - donc juste à bâtir FTDataset(train=nimportequoi, test, valid=pNIST etc.)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
87 # - on veut dans l'array mettre des pointeurs vers la fonction either test ou valid
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
88 # donc PAS dataset_obj, mais dataset_obj.train (sans les parenthèses)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
89 def build_test_valid_sets():
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
90 nist_ds = datasets.nist_all()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
91 pnist_ds = datasets.PNIST07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
92 p07_ds = datasets.nist_P07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
93
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
94 test_valid_fns = [nist_ds.test, nist_ds.valid,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
95 pnist_ds.test, pnist_ds.valid,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
96 p07_ds.test, p07_ds.valid]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
97
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
98 test_valid_names = ["nist_all__test", "nist_all__valid",
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
99 "NISTP__test", "NISTP__valid",
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
100 "P07__test", "P07__valid"]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
101
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
102 return test_valid_fns, test_valid_names
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
103
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
104 def add_error_series(series, error_name, hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
105 index_names=('minibatch_idx',), use_accumulator=False,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
106 reduce_every=250):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
107 # train
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
108 series_base = ErrorSeries(error_name=error_name,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
109 table_name=error_name,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
110 hdf5_file=hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
111 index_names=index_names)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
112
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
113 if use_accumulator:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
114 series[error_name] = \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
115 AccumulatorSeriesWrapper(base_series=series_base,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
116 reduce_every=reduce_every)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
117 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
118 series[error_name] = series_base
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
119
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
120 TEST_VALID_FNS,TEST_VALID_NAMES = None, None
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
121 def compute_and_save_errors(state, mlp, series, hdf5_file, minibatch_idx):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
122 global TEST_VALID_FNS,TEST_VALID_NAMES
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
123
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
124 TEST_VALID_FNS,TEST_VALID_NAMES = build_test_valid_sets()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
125
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
126 # if the training is on digits only, then there'll be a 100%
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
127 # error on digits in the valid/test set... just ignore them
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
128
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
129 test_fn = theano.function([mlp.input], mlp.logRegressionLayer.y_pred)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
130
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
131 test_batch_size = 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
132 for test_ds_fn,test_ds_name in zip(TEST_VALID_FNS,TEST_VALID_NAMES):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
133 # reset error counts for every test/valid set
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
134 # note: float
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
135 total_errors = total_digit_errors = \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
136 total_uppercase_errors = total_lowercase_errors = 0.
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
137
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
138 total_all = total_lowercase = total_uppercase = total_digit = 0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
139
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
140 for mb_x,mb_y in test_ds_fn(test_batch_size):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
141 digit_mask = mb_y < 10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
142 uppercase_mask = mb_y >= 36
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
143 lowercase_mask = numpy.ones((len(mb_x),)) \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
144 - digit_mask - uppercase_mask
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
145
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
146 total_all += len(mb_x)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
147 total_digit += sum(digit_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
148 total_uppercase += sum(uppercase_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
149 total_lowercase += sum(lowercase_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
150
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
151 predictions = test_fn(mb_x)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
152
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
153 all_errors = (mb_y != predictions)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
154 total_errors += sum(all_errors)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
155
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
156 if len(all_errors) != len(digit_mask):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
157 print "size all", all_errors.shape, " digit", digit_mask.shape
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
158 total_digit_errors += sum(numpy.multiply(all_errors, digit_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
159 total_uppercase_errors += sum(numpy.multiply(all_errors, uppercase_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
160 total_lowercase_errors += sum(numpy.multiply(all_errors, lowercase_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
161
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
162 four_errors = [float(total_errors) / total_all,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
163 float(total_digit_errors) / total_digit,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
164 float(total_lowercase_errors) / total_lowercase,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
165 float(total_uppercase_errors) / total_uppercase]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
166
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
167 four_errors_names = ["all", "digits", "lower", "upper"]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
168
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
169 # record stats per set
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
170 print "Errors on", test_ds_name, ",".join(four_errors_names),\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
171 ":", ",".join([str(e) for e in four_errors])
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
172
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
173 # now in the state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
174 for err, errname in zip(four_errors, four_errors_names):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
175 error_full_name = 'error__'+test_ds_name+'_'+errname
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
176 min_name = 'min_'+error_full_name
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
177 minpos_name = 'minpos_'+error_full_name
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
178
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
179 if state.has_key(min_name):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
180 if state[min_name] > err:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
181 state[min_name] = err
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
182 state[minpos_name] = pos_str
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
183 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
184 # also create the series
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
185 add_error_series(series, error_full_name, hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
186 index_names=('minibatch_idx',))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
187 state[min_name] = err
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
188 state[minpos_name] = minibatch_idx
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
189
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
190 state[minpos_name] = pos_str
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
191 series[error_full_name].append((minibatch_idx,), err)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
192
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
193 def jobman_entrypoint(state, channel):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
194 global TEST_RUN
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
195 minibatch_size = state.minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
196
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
197 print_every = 100000
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
198 COMPUTE_ERROR_EVERY = 10**7 / minibatch_size # compute error every 10 million examples
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
199 if TEST_RUN:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
200 print_every = 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
201 COMPUTE_ERROR_EVERY = 1000 / minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
202
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
203 print "entrypoint, state is"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
204 print state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
205
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
206 ######################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
207 # select dataset and dataset subset, plus adjust epoch num to make number
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
208 # of examples seen independent of dataset
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
209 # exemple: pour le cas DIGITS_ONLY, il faut changer le nombre d'époques
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
210 # et pour le cas NIST pur (pas de transformations), il faut multiplier par 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
211 # en partant car on a pas les variations
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
212
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
213 # compute this in terms of the P07 dataset size (=80M)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
214 MINIBATCHES_TO_SEE = state.n_epochs * 8 * (10**6) / minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
215
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
216 if state.train_on == 'NIST' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
217 dataset_obj = datasets.nist_all()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
218 elif state.train_on == 'NIST' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
219 dataset_obj = datasets.nist_digits()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
220 elif state.train_on == 'NISTP' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
221 dataset_obj = datasets.PNIST07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
222 elif state.train_on == 'NISTP' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
223 dataset_obj = PNIST07_digits
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
224 elif state.train_on == 'P07' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
225 dataset_obj = datasets.nist_P07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
226 elif state.train_on == 'P07' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
227 dataset_obj = datasets.P07_digits
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
228
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
229 dataset = dataset_obj
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
230
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
231 if state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
232 n_classes = 62
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
233 elif state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
234 n_classes = 10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
235 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
236 raise NotImplementedError()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
237
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
238 ###############################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
239 # construct model
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
240
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
241 print "constructing model..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
242 x = T.matrix('x')
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
243 y = T.ivector('y')
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
244
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
245 rng = numpy.random.RandomState(state.rng_seed)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
246
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
247 # construct the MLP class
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
248 model = MLP(rng = rng, input=x, n_in=N_INPUTS,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
249 n_hidden_layers = state.n_hidden_layers,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
250 n_hidden = state.n_hidden, n_out=n_classes)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
251
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
252
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
253 # cost and training fn
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
254 cost = T.mean(model.negative_log_likelihood(y)) \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
255 + state.L1_reg * model.L1 \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
256 + state.L2_reg * model.L2_sqr
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
257
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
258 print "L1, L2: ", state.L1_reg, state.L2_reg
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
259
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
260 gradient_nll_wrt_params = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
261 for param in model.params:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
262 gparam = T.grad(cost, param)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
263 gradient_nll_wrt_params.append(gparam)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
264
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
265 learning_rate = 10**float(state.learning_rate_log10)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
266 print "Learning rate", learning_rate
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
267
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
268 train_updates = {}
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
269 for param, gparam in zip(model.params, gradient_nll_wrt_params):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
270 train_updates[param] = param - learning_rate * gparam
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
271
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
272 train_fn = theano.function([x,y], cost, updates=train_updates)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
273
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
274 #######################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
275 # create series
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
276 basedir = os.getcwd()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
277
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
278 h5f = tables.openFile(os.path.join(basedir, "series.h5"), "w")
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
279
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
280 series = {}
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
281 add_error_series(series, "training_error", h5f,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
282 index_names=('minibatch_idx',), use_accumulator=True,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
283 reduce_every=REDUCE_EVERY)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
284
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
285 ##########################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
286 # training loop
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
287
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
288 start_time = time.clock()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
289
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
290 print "begin training..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
291 print "will train for", MINIBATCHES_TO_SEE, "examples"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
292
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
293 mb_idx = 0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
294
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
295 while(mb_idx*minibatch_size<nb_max_exemples):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
296
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
297 last_costs = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
298
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
299 for mb_x, mb_y in dataset.train(minibatch_size):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
300 if TEST_RUN and mb_idx > 1000:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
301 break
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
302
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
303 last_cost = train_fn(mb_x, mb_y)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
304 series["training_error"].append((mb_idx,), last_cost)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
305
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
306 last_costs.append(last_cost)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
307 if (len(last_costs)+1) % print_every == 0:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
308 print "Mean over last", print_every, "minibatches: ", numpy.mean(last_costs)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
309 last_costs = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
310
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
311 if (mb_idx+1) % COMPUTE_ERROR_EVERY == 0:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
312 # compute errors
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
313 print "computing errors on all datasets..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
314 print "Time since training began: ", (time.clock()-start_time)/60., "minutes"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
315 compute_and_save_errors(state, model, series, h5f, mb_idx)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
316
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
317 channel.save()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
318
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
319 sys.stdout.flush()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
320
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
321 end_time = time.clock()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
322
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
323 print "-"*80
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
324 print "Finished. Training took", (end_time-start_time)/60., "minutes"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
325 print state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
326
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
327 def run_test():
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
328 global TEST_RUN
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
329 from fsml.job_management import mock_channel
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
330 TEST_RUN = True
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
331 jobman_entrypoint(TEST_HP, mock_channel)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
332
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
333 if __name__ == '__main__':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
334 run_test()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
335