Mercurial > ift6266

annotate deep/crbm/mnist_crbm.py @ 339:ffbf0e41bcee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
author fsavard
date Sat, 17 Apr 2010 20:29:18 -0400
parents 8d116d4a7593
children 523e7b87c521
rev   line source
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
1 #!/usr/bin/python
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
2
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
3 # do this first
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
4 from mnist_config import *
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
5
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
6 import sys
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
7 import os, os.path
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
8
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
9 import numpy as N
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
10
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
11 import theano
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
12 import theano.tensor as T
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
13
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
14 from crbm import CRBM, ConvolutionParams
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
15
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
16 from pylearn.datasets import MNIST
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
17 from pylearn.io.image_tiling import tile_raster_images
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
18
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
19 import Image
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
20
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
21 from pylearn.io.seriestables import *
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
22 import tables
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
23
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
24 import utils
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
25
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
26 #def filename_from_time(suffix):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
27 # import datetime
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
28 # return str(datetime.datetime.now()) + suffix + ".png"
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
29
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
30 def jobman_entrypoint(state, channel):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
31 # record mercurial versions of each package
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
32 pylearn.version.record_versions(state,[theano,ift6266,pylearn])
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
33 channel.save()
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
34
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
35 crbm = MnistCrbm(state)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
36 crbm.train()
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
37
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
38 return channel.COMPLETE
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
39
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
40 class MnistCrbm(object):
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
41 def __init__(self, state):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
42 self.state = state
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
43
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
44 if TEST_CONFIG:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
45 self.mnist = MNIST.full()#first_10k()
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
46
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
47 self.cp = ConvolutionParams( \
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
48 num_filters=state.num_filters,
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
49 num_input_planes=1,
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
50 height_filters=state.filter_size,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
51 width_filters=state.filter_size)
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
52
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
53 self.image_size = (28,28)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
54
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
55 self.minibatch_size = state.minibatch_size
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
56
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
57 self.lr = state.learning_rate
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
58 self.sparsity_lambda = state.sparsity_lambda
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
59 # about 1/num_filters, so only one filter active at a time
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
60 # 40 * 0.05 = ~2 filters active for any given pixel
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
61 self.sparsity_p = state.sparsity_p
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
62
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
63 self.crbm = CRBM( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
64 minibatch_size=self.minibatch_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
65 image_size=self.image_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
66 conv_params=self.cp,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
67 learning_rate=self.lr,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
68 sparsity_lambda=self.sparsity_lambda,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
69 sparsity_p=self.sparsity_p)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
70
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
71 self.num_epochs = state.num_epochs
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
72
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
73 self.init_series()
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
74
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
75 def init_series(self):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
76 series = {}
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
77
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
78 basedir = os.getcwd()
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
79
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
80 h5f = tables.openFile(os.path.join(basedir, "series.h5"), "w")
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
81
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
82 cd_series_names = self.crbm.cd_return_desc
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
83 series['cd'] = \
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
84 utils.get_accumulator_series_array( \
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
85 h5f, 'cd', cd_series_names,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
86 REDUCE_EVERY,
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
87 stdout_too=SERIES_STDOUT_TOO)
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
88
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
89 sparsity_series_names = self.crbm.sparsity_return_desc
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
90 series['sparsity'] = \
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
91 utils.get_accumulator_series_array( \
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
92 h5f, 'sparsity', sparsity_series_names,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
93 REDUCE_EVERY,
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
94 stdout_too=SERIES_STDOUT_TOO)
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
95
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
96 # so first we create the names for each table, based on
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
97 # position of each param in the array
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
98 params_stdout = []
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
99 if SERIES_STDOUT_TOO:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
100 params_stdout = [StdoutAppendTarget()]
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
101 series['params'] = SharedParamsStatisticsWrapper(
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
102 new_group_name="params",
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
103 base_group="/",
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
104 arrays_names=['W','b_h','b_x'],
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
105 hdf5_file=h5f,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
106 index_names=('epoch','minibatch'),
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
107 other_targets=params_stdout)
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
108
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
109 self.series = series
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
110
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
111 def train(self):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
112 num_minibatches = len(self.mnist.train.x) / self.minibatch_size
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
113
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
114 for epoch in xrange(self.num_epochs):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
115 for mb_index in xrange(num_minibatches):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
116 mb_x = self.mnist.train.x \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
117 [mb_index : mb_index+self.minibatch_size]
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
118 mb_x = mb_x.reshape((self.minibatch_size, 1, 28, 28))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
119
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
120 #E_h = crbm.E_h_given_x_func(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
121 #print "Shape of E_h", E_h.shape
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
122
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
123 cd_return = self.crbm.CD_step(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
124 sp_return = self.crbm.sparsity_step(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
125
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
126 self.series['cd'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
127 (epoch, mb_index), cd_return)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
128 self.series['sparsity'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
129 (epoch, mb_index), sp_return)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
130
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
131 total_idx = epoch*num_minibatches + mb_index
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
132
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
133 if (total_idx+1) % REDUCE_EVERY == 0:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
134 self.series['params'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
135 (epoch, mb_index), self.crbm.params)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
136
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
137 if total_idx % VISUALIZE_EVERY == 0:
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
138 self.visualize_gibbs_result(\
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
139 mb_x, GIBBS_STEPS_IN_VIZ_CHAIN,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
140 "gibbs_chain_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
141 self.visualize_gibbs_result(mb_x, 1,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
142 "gibbs_1_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
143 self.visualize_filters(
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
144 "filters_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
145 if TEST_CONFIG:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
146 # do a single epoch for cluster tests config
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
147 break
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
148
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
149 if SAVE_PARAMS:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
150 utils.save_params(self.crbm.params, "params.pkl")
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
151
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
152 def visualize_gibbs_result(self, start_x, gibbs_steps, filename):
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
153 # Run minibatch_size chains for gibbs_steps
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
154 x_samples = None
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
155 if not start_x is None:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
156 x_samples = self.crbm.gibbs_samples_from(start_x, gibbs_steps)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
157 else:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
158 x_samples = self.crbm.random_gibbs_samples(gibbs_steps)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
159 x_samples = x_samples.reshape((self.minibatch_size, 28*28))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
160
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
161 tile = tile_raster_images(x_samples, self.image_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
162 (1, self.minibatch_size), output_pixel_vals=True)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
163
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
164 filepath = os.path.join(IMAGE_OUTPUT_DIR, filename+".png")
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
165 img = Image.fromarray(tile)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
166 img.save(filepath)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
167
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
168 print "Result of running Gibbs", \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
169 gibbs_steps, "times outputed to", filepath
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
170
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
171 def visualize_filters(self, filename):
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
172 cp = self.cp
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
173
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
174 # filter size
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
175 fsz = (cp.height_filters, cp.width_filters)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
176 tile_shape = (cp.num_filters, cp.num_input_planes)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
177
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
178 filters_flattened = self.crbm.W.value.reshape(
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
179 (tile_shape[0]*tile_shape[1],
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
180 fsz[0]*fsz[1]))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
181
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
182 tile = tile_raster_images(filters_flattened, fsz,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
183 tile_shape, output_pixel_vals=True)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
184
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
185 filepath = os.path.join(IMAGE_OUTPUT_DIR, filename+".png")
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
186 img = Image.fromarray(tile)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
187 img.save(filepath)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
188
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
189 print "Filters (as images) outputed to", filepath
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
190
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
191
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
192
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
193 if __name__ == '__main__':
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
194 args = sys.argv[1:]
337
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet.
fsavard
parents:
diff changeset
195
339
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
196 if len(args) == 0:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
197 print "Bad usage"
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
198 elif args[0] == 'jobman_insert':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
199 utils.jobman_insert_job_vals(JOBDB, EXPERIMENT_PATH, JOB_VALS)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
200 elif args[0] == 'test_jobman_entrypoint':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
201 chanmock = DD({'COMPLETE':0,'save':(lambda:None)})
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
202 jobman_entrypoint(DEFAULT_STATE, chanmock)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
203 elif args[0] == 'run_default':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
204 mc = MnistCrbm(DEFAULT_STATE)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
205 mc.train()
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
206
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet.
fsavard
parents: 337
diff changeset
207