Mercurial > ift6266

annotate deep/deep_mlp/job.py @ 626:75dbbe409578

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
author fsavard
date Wed, 16 Mar 2011 13:43:32 -0400
parents
children
rev   line source
626
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
1 #!/usr/bin/env python
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
2 # coding: utf-8
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
3
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
4 '''
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
5 Launching
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
6
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
7 jobman sqlschedules postgres://ift6266h10@gershwin/ift6266h10_sandbox_db/mlp_dumi mlp_jobman.experiment mlp_jobman.conf
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
8 'n_hidden={{500,1000,2000}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
9 'n_hidden_layers={{2,3}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
10 'train_on={{NIST,NISTP,P07}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
11 'train_subset={{DIGITS_ONLY,ALL}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
12 'learning_rate_log10={{-1.,-2.,-3.}}'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
13
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
14 in mlp_jobman.conf:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
15 rng_seed=1234
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
16 L1_reg=0.0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
17 L2_reg=0.0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
18 n_epochs=10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
19 minibatch_size=20
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
20 '''
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
21
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
22 import os, sys, copy, operator, time
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
23 import theano
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
24 import theano.tensor as T
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
25 import numpy
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
26 from mlp import MLP
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
27 from ift6266 import datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
28 from pylearn.io.seriestables import *
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
29 import tables
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
30 from jobman.tools import DD
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
31
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
32 N_INPUTS = 32*32
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
33 REDUCE_EVERY = 250
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
34
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
35 TEST_RUN = False
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
36
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
37 TEST_HP = DD({'n_hidden':200,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
38 'n_hidden_layers': 2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
39 'train_on':'NIST',
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
40 'train_subset':'ALL',
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
41 'learning_rate_log10':-2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
42 'rng_seed':1234,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
43 'L1_reg':0.0,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
44 'L2_reg':0.0,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
45 'n_epochs':2,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
46 'minibatch_size':20})
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
47
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
48 ###########################################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
49 # digits datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
50 # nist_digits is already in NIST_PATH and in ift6266.datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
51 # NOTE: for these datasets the test and valid sets are wrong
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
52 # (don't correspond to the training set... they're just placeholders)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
53
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
54 from ift6266.datasets.defs import NIST_PATH, DATA_PATH
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
55 TRANSFORMED_DIGITS_PATH = '/data/lisatmp/ift6266h10/data/transformed_digits'
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
56
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
57 P07_digits = FTDataSet(\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
58 train_data = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
59 'data/P07_train'+str(i)+'_data.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
60 for i in range(0, 100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
61 train_lbl = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
62 'data/P07_train'+str(i)+'_labels.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
63 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
64 test_data = [os.path.join(DATA_PATH,'data/P07_test_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
65 test_lbl = [os.path.join(DATA_PATH,'data/P07_test_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
66 valid_data = [os.path.join(DATA_PATH,'data/P07_valid_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
67 valid_lbl = [os.path.join(DATA_PATH,'data/P07_valid_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
68 indtype=theano.config.floatX, inscale=255., maxsize=None)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
69
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
70 #Added PNIST
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
71 PNIST07_digits = FTDataSet(train_data = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
72 'PNIST07_train'+str(i)+'_data.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
73 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
74 train_lbl = [os.path.join(TRANSFORMED_DIGITS_PATH,\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
75 'PNIST07_train'+str(i)+'_labels.ft')\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
76 for i in range(0,100)],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
77 test_data = [os.path.join(DATA_PATH,'data/PNIST07_test_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
78 test_lbl = [os.path.join(DATA_PATH,'data/PNIST07_test_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
79 valid_data = [os.path.join(DATA_PATH,'data/PNIST07_valid_data.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
80 valid_lbl = [os.path.join(DATA_PATH,'data/PNIST07_valid_labels.ft')],
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
81 indtype=theano.config.floatX, inscale=255., maxsize=None)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
82
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
83
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
84 # building valid_test_datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
85 # - on veut des dataset_obj pour les 3 datasets
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
86 # - donc juste à bâtir FTDataset(train=nimportequoi, test, valid=pNIST etc.)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
87 # - on veut dans l'array mettre des pointeurs vers la fonction either test ou valid
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
88 # donc PAS dataset_obj, mais dataset_obj.train (sans les parenthèses)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
89 def build_test_valid_sets():
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
90 nist_ds = datasets.nist_all()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
91 pnist_ds = datasets.PNIST07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
92 p07_ds = datasets.nist_P07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
93
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
94 test_valid_fns = [nist_ds.test, nist_ds.valid,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
95 pnist_ds.test, pnist_ds.valid,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
96 p07_ds.test, p07_ds.valid]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
97
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
98 test_valid_names = ["nist_all__test", "nist_all__valid",
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
99 "NISTP__test", "NISTP__valid",
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
100 "P07__test", "P07__valid"]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
101
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
102 return test_valid_fns, test_valid_names
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
103
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
104 def add_error_series(series, error_name, hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
105 index_names=('minibatch_idx',), use_accumulator=False,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
106 reduce_every=250):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
107 # train
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
108 series_base = ErrorSeries(error_name=error_name,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
109 table_name=error_name,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
110 hdf5_file=hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
111 index_names=index_names)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
112
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
113 if use_accumulator:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
114 series[error_name] = \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
115 AccumulatorSeriesWrapper(base_series=series_base,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
116 reduce_every=reduce_every)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
117 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
118 series[error_name] = series_base
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
119
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
120 TEST_VALID_FNS,TEST_VALID_NAMES = None, None
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
121 def compute_and_save_errors(state, mlp, series, hdf5_file, minibatch_idx):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
122 global TEST_VALID_FNS,TEST_VALID_NAMES
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
123
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
124 TEST_VALID_FNS,TEST_VALID_NAMES = build_test_valid_sets()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
125
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
126 # if the training is on digits only, then there'll be a 100%
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
127 # error on digits in the valid/test set... just ignore them
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
128
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
129 test_fn = theano.function([mlp.input], mlp.logRegressionLayer.y_pred)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
130
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
131 test_batch_size = 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
132 for test_ds_fn,test_ds_name in zip(TEST_VALID_FNS,TEST_VALID_NAMES):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
133 # reset error counts for every test/valid set
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
134 # note: float
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
135 total_errors = total_digit_errors = \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
136 total_uppercase_errors = total_lowercase_errors = 0.
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
137
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
138 total_all = total_lowercase = total_uppercase = total_digit = 0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
139
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
140 for mb_x,mb_y in test_ds_fn(test_batch_size):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
141 digit_mask = mb_y < 10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
142 uppercase_mask = mb_y >= 36
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
143 lowercase_mask = numpy.ones((len(mb_x),)) \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
144 - digit_mask - uppercase_mask
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
145
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
146 total_all += len(mb_x)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
147 total_digit += sum(digit_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
148 total_uppercase += sum(uppercase_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
149 total_lowercase += sum(lowercase_mask)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
150
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
151 predictions = test_fn(mb_x)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
152
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
153 all_errors = (mb_y != predictions)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
154 total_errors += sum(all_errors)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
155
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
156 if len(all_errors) != len(digit_mask):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
157 print "size all", all_errors.shape, " digit", digit_mask.shape
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
158 total_digit_errors += sum(numpy.multiply(all_errors, digit_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
159 total_uppercase_errors += sum(numpy.multiply(all_errors, uppercase_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
160 total_lowercase_errors += sum(numpy.multiply(all_errors, lowercase_mask))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
161
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
162 four_errors = [float(total_errors) / total_all,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
163 float(total_digit_errors) / total_digit,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
164 float(total_lowercase_errors) / total_lowercase,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
165 float(total_uppercase_errors) / total_uppercase]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
166
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
167 four_errors_names = ["all", "digits", "lower", "upper"]
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
168
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
169 # record stats per set
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
170 print "Errors on", test_ds_name, ",".join(four_errors_names),\
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
171 ":", ",".join([str(e) for e in four_errors])
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
172
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
173 # now in the state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
174 for err, errname in zip(four_errors, four_errors_names):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
175 error_full_name = 'error__'+test_ds_name+'_'+errname
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
176 min_name = 'min_'+error_full_name
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
177 minpos_name = 'minpos_'+error_full_name
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
178
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
179 if state.has_key(min_name):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
180 if state[min_name] > err:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
181 state[min_name] = err
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
182 state[minpos_name] = pos_str
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
183 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
184 # also create the series
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
185 add_error_series(series, error_full_name, hdf5_file,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
186 index_names=('minibatch_idx',))
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
187 state[min_name] = err
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
188 state[minpos_name] = minibatch_idx
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
189
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
190 state[minpos_name] = pos_str
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
191 series[error_full_name].append((minibatch_idx,), err)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
192
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
193 def jobman_entrypoint(state, channel):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
194 global TEST_RUN
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
195 minibatch_size = state.minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
196
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
197 print_every = 100000
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
198 COMPUTE_ERROR_EVERY = 10**7 / minibatch_size # compute error every 10 million examples
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
199 if TEST_RUN:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
200 print_every = 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
201 COMPUTE_ERROR_EVERY = 1000 / minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
202
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
203 print "entrypoint, state is"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
204 print state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
205
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
206 ######################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
207 # select dataset and dataset subset, plus adjust epoch num to make number
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
208 # of examples seen independent of dataset
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
209 # exemple: pour le cas DIGITS_ONLY, il faut changer le nombre d'époques
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
210 # et pour le cas NIST pur (pas de transformations), il faut multiplier par 100
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
211 # en partant car on a pas les variations
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
212
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
213 # compute this in terms of the P07 dataset size (=80M)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
214 MINIBATCHES_TO_SEE = state.n_epochs * 8 * (10**6) / minibatch_size
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
215
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
216 if state.train_on == 'NIST' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
217 dataset_obj = datasets.nist_all()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
218 elif state.train_on == 'NIST' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
219 dataset_obj = datasets.nist_digits()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
220 elif state.train_on == 'NISTP' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
221 dataset_obj = datasets.PNIST07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
222 elif state.train_on == 'NISTP' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
223 dataset_obj = PNIST07_digits
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
224 elif state.train_on == 'P07' and state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
225 dataset_obj = datasets.nist_P07()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
226 elif state.train_on == 'P07' and state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
227 dataset_obj = datasets.P07_digits
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
228
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
229 dataset = dataset_obj
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
230
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
231 if state.train_subset == 'ALL':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
232 n_classes = 62
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
233 elif state.train_subset == 'DIGITS_ONLY':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
234 n_classes = 10
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
235 else:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
236 raise NotImplementedError()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
237
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
238 ###############################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
239 # construct model
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
240
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
241 print "constructing model..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
242 x = T.matrix('x')
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
243 y = T.ivector('y')
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
244
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
245 rng = numpy.random.RandomState(state.rng_seed)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
246
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
247 # construct the MLP class
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
248 model = MLP(rng = rng, input=x, n_in=N_INPUTS,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
249 n_hidden_layers = state.n_hidden_layers,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
250 n_hidden = state.n_hidden, n_out=n_classes)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
251
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
252
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
253 # cost and training fn
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
254 cost = T.mean(model.negative_log_likelihood(y)) \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
255 + state.L1_reg * model.L1 \
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
256 + state.L2_reg * model.L2_sqr
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
257
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
258 print "L1, L2: ", state.L1_reg, state.L2_reg
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
259
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
260 gradient_nll_wrt_params = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
261 for param in model.params:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
262 gparam = T.grad(cost, param)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
263 gradient_nll_wrt_params.append(gparam)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
264
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
265 learning_rate = 10**float(state.learning_rate_log10)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
266 print "Learning rate", learning_rate
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
267
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
268 train_updates = {}
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
269 for param, gparam in zip(model.params, gradient_nll_wrt_params):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
270 train_updates[param] = param - learning_rate * gparam
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
271
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
272 train_fn = theano.function([x,y], cost, updates=train_updates)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
273
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
274 #######################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
275 # create series
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
276 basedir = os.getcwd()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
277
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
278 h5f = tables.openFile(os.path.join(basedir, "series.h5"), "w")
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
279
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
280 series = {}
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
281 add_error_series(series, "training_error", h5f,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
282 index_names=('minibatch_idx',), use_accumulator=True,
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
283 reduce_every=REDUCE_EVERY)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
284
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
285 ##########################
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
286 # training loop
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
287
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
288 start_time = time.clock()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
289
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
290 print "begin training..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
291 print "will train for", MINIBATCHES_TO_SEE, "examples"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
292
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
293 mb_idx = 0
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
294
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
295 while(mb_idx*minibatch_size<nb_max_exemples):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
296
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
297 last_costs = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
298
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
299 for mb_x, mb_y in dataset.train(minibatch_size):
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
300 if TEST_RUN and mb_idx > 1000:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
301 break
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
302
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
303 last_cost = train_fn(mb_x, mb_y)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
304 series["training_error"].append((mb_idx,), last_cost)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
305
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
306 last_costs.append(last_cost)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
307 if (len(last_costs)+1) % print_every == 0:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
308 print "Mean over last", print_every, "minibatches: ", numpy.mean(last_costs)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
309 last_costs = []
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
310
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
311 if (mb_idx+1) % COMPUTE_ERROR_EVERY == 0:
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
312 # compute errors
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
313 print "computing errors on all datasets..."
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
314 print "Time since training began: ", (time.clock()-start_time)/60., "minutes"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
315 compute_and_save_errors(state, model, series, h5f, mb_idx)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
316
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
317 channel.save()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
318
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
319 sys.stdout.flush()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
320
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
321 end_time = time.clock()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
322
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
323 print "-"*80
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
324 print "Finished. Training took", (end_time-start_time)/60., "minutes"
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
325 print state
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
326
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
327 def run_test():
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
328 global TEST_RUN
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
329 from fsml.job_management import mock_channel
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
330 TEST_RUN = True
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
331 jobman_entrypoint(TEST_HP, mock_channel)
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
332
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
333 if __name__ == '__main__':
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
334 run_test()
75dbbe409578 Added code for deep mlp, experiment code to go along with it. Also added code I used to filter the P07 / PNIST07 datasets to keep only digits.
fsavard
parents:
diff changeset
335