ift6266: deep/crbm/mnist_crbm.py annotate

annotate deep/crbm/mnist_crbm.py @ 343:82dae7c46046

Last few bugfixes before launching

author	fsavard
date	Sun, 18 Apr 2010 11:54:57 -0400
parents	b25ad1670ff7
children	ed1c0830681e

rev	line source
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	1 #!/usr/bin/python
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	2
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	3 import sys
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	4
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	5 # do this before importing custom modules
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	6 from mnist_config import *
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	7
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	8 if not (len(sys.argv) > 1 and sys.argv[1] in \
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	9 ('test_jobman_entrypoint', 'run_local')):
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	10 # in those cases don't use isolated code, use dev code
342 b25ad1670ff7 Slight bug fsavard parents: 340 diff changeset	11 isolate_experiment()
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	12
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	13 import os, os.path
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	14
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	15 import numpy as N
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	16
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	17 import theano
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	18 import theano.tensor as T
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	19
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	20 from crbm import CRBM, ConvolutionParams
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	21
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	22 import pylearn, pylearn.version
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	23 from pylearn.datasets import MNIST
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	24 from pylearn.io.image_tiling import tile_raster_images
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	25
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	26 import Image
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	27
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	28 from pylearn.io.seriestables import *
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	29 import tables
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	30
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	31 import ift6266
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	32
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	33 import utils
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	34
343 82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	35 if not (len(sys.argv) > 1 and sys.argv[1] in \
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	36 ('jobman_insert',)):
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	37 if not os.path.exists(IMAGE_OUTPUT_DIR):
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	38 os.mkdir(IMAGE_OUTPUT_DIR)
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	39 elif os.path.isfile(IMAGE_OUTPUT_DIR):
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	40 print "IMAGE_OUTPUT_DIR is not a directory!"
82dae7c46046 Last few bugfixes before launching fsavard parents: 342 diff changeset	41 sys.exit(1)
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	42
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	43 #def filename_from_time(suffix):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	44 # import datetime
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	45 # return str(datetime.datetime.now()) + suffix + ".png"
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	46
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	47 def jobman_entrypoint(state, channel):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	48 # record mercurial versions of each package
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	49 pylearn.version.record_versions(state,[theano,ift6266,pylearn])
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	50 channel.save()
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	51
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	52 crbm = MnistCrbm(state)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	53 crbm.train()
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	54
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	55 return channel.COMPLETE
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	56
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	57 class MnistCrbm(object):
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	58 def __init__(self, state):
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	59 self.state = state
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	60
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	61 if TEST_CONFIG:
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	62 self.mnist = MNIST.first_1k()
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	63 print "Test config, so loaded MNIST first 1000"
523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	64 else:
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	65 self.mnist = MNIST.full()#first_10k()
340 523e7b87c521 Corrected a few bugs, no new features. Supposedly ready to run on cluster. fsavard parents: 339 diff changeset	66 print "Loaded MNIST full"
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	67
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	68 self.cp = ConvolutionParams( \
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	69 num_filters=state.num_filters,
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	70 num_input_planes=1,
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	71 height_filters=state.filter_size,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	72 width_filters=state.filter_size)
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	73
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	74 self.image_size = (28,28)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	75
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	76 self.minibatch_size = state.minibatch_size
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	77
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	78 self.lr = state.learning_rate
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	79 self.sparsity_lambda = state.sparsity_lambda
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	80 # about 1/num_filters, so only one filter active at a time
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	81 # 40 * 0.05 = ~2 filters active for any given pixel
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	82 self.sparsity_p = state.sparsity_p
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	83
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	84 self.crbm = CRBM( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	85 minibatch_size=self.minibatch_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	86 image_size=self.image_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	87 conv_params=self.cp,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	88 learning_rate=self.lr,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	89 sparsity_lambda=self.sparsity_lambda,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	90 sparsity_p=self.sparsity_p)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	91
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	92 self.num_epochs = state.num_epochs
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	93
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	94 self.init_series()
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	95
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	96 def init_series(self):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	97 series = {}
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	98
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	99 basedir = os.getcwd()
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	100
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	101 h5f = tables.openFile(os.path.join(basedir, "series.h5"), "w")
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	102
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	103 cd_series_names = self.crbm.cd_return_desc
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	104 series['cd'] = \
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	105 utils.get_accumulator_series_array( \
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	106 h5f, 'cd', cd_series_names,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	107 REDUCE_EVERY,
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	108 stdout_too=SERIES_STDOUT_TOO)
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	109
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	110 sparsity_series_names = self.crbm.sparsity_return_desc
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	111 series['sparsity'] = \
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	112 utils.get_accumulator_series_array( \
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	113 h5f, 'sparsity', sparsity_series_names,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	114 REDUCE_EVERY,
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	115 stdout_too=SERIES_STDOUT_TOO)
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	116
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	117 # so first we create the names for each table, based on
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	118 # position of each param in the array
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	119 params_stdout = []
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	120 if SERIES_STDOUT_TOO:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	121 params_stdout = [StdoutAppendTarget()]
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	122 series['params'] = SharedParamsStatisticsWrapper(
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	123 new_group_name="params",
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	124 base_group="/",
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	125 arrays_names=['W','b_h','b_x'],
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	126 hdf5_file=h5f,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	127 index_names=('epoch','minibatch'),
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	128 other_targets=params_stdout)
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	129
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	130 self.series = series
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	131
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	132 def train(self):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	133 num_minibatches = len(self.mnist.train.x) / self.minibatch_size
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	134
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	135 for epoch in xrange(self.num_epochs):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	136 for mb_index in xrange(num_minibatches):
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	137 mb_x = self.mnist.train.x \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	138 [mb_index : mb_index+self.minibatch_size]
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	139 mb_x = mb_x.reshape((self.minibatch_size, 1, 28, 28))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	140
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	141 #E_h = crbm.E_h_given_x_func(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	142 #print "Shape of E_h", E_h.shape
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	143
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	144 cd_return = self.crbm.CD_step(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	145 sp_return = self.crbm.sparsity_step(mb_x)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	146
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	147 self.series['cd'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	148 (epoch, mb_index), cd_return)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	149 self.series['sparsity'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	150 (epoch, mb_index), sp_return)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	151
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	152 total_idx = epoch*num_minibatches + mb_index
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	153
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	154 if (total_idx+1) % REDUCE_EVERY == 0:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	155 self.series['params'].append( \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	156 (epoch, mb_index), self.crbm.params)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	157
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	158 if total_idx % VISUALIZE_EVERY == 0:
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	159 self.visualize_gibbs_result(\
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	160 mb_x, GIBBS_STEPS_IN_VIZ_CHAIN,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	161 "gibbs_chain_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	162 self.visualize_gibbs_result(mb_x, 1,
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	163 "gibbs_1_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	164 self.visualize_filters(
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	165 "filters_"+str(epoch)+"_"+str(mb_index))
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	166 if TEST_CONFIG:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	167 # do a single epoch for cluster tests config
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	168 break
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	169
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	170 if SAVE_PARAMS:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	171 utils.save_params(self.crbm.params, "params.pkl")
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	172
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	173 def visualize_gibbs_result(self, start_x, gibbs_steps, filename):
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	174 # Run minibatch_size chains for gibbs_steps
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	175 x_samples = None
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	176 if not start_x is None:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	177 x_samples = self.crbm.gibbs_samples_from(start_x, gibbs_steps)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	178 else:
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	179 x_samples = self.crbm.random_gibbs_samples(gibbs_steps)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	180 x_samples = x_samples.reshape((self.minibatch_size, 28*28))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	181
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	182 tile = tile_raster_images(x_samples, self.image_size,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	183 (1, self.minibatch_size), output_pixel_vals=True)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	184
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	185 filepath = os.path.join(IMAGE_OUTPUT_DIR, filename+".png")
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	186 img = Image.fromarray(tile)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	187 img.save(filepath)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	188
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	189 print "Result of running Gibbs", \
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	190 gibbs_steps, "times outputed to", filepath
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	191
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	192 def visualize_filters(self, filename):
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	193 cp = self.cp
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	194
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	195 # filter size
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	196 fsz = (cp.height_filters, cp.width_filters)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	197 tile_shape = (cp.num_filters, cp.num_input_planes)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	198
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	199 filters_flattened = self.crbm.W.value.reshape(
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	200 (tile_shape[0]*tile_shape[1],
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	201 fsz[0]*fsz[1]))
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	202
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	203 tile = tile_raster_images(filters_flattened, fsz,
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	204 tile_shape, output_pixel_vals=True)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	205
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	206 filepath = os.path.join(IMAGE_OUTPUT_DIR, filename+".png")
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	207 img = Image.fromarray(tile)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	208 img.save(filepath)
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	209
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	210 print "Filters (as images) outputed to", filepath
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	211
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	212
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	213
8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	214 if __name__ == '__main__':
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	215 args = sys.argv[1:]
337 8d116d4a7593 Added convolutional RBM (ala Lee09) code, imported from my working dir elsewhere. Seems to work for one layer. No subsampling yet. fsavard parents: diff changeset	216
339 ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	217 if len(args) == 0:
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	218 print "Bad usage"
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	219 elif args[0] == 'jobman_insert':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	220 utils.jobman_insert_job_vals(JOBDB, EXPERIMENT_PATH, JOB_VALS)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	221 elif args[0] == 'test_jobman_entrypoint':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	222 chanmock = DD({'COMPLETE':0,'save':(lambda:None)})
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	223 jobman_entrypoint(DEFAULT_STATE, chanmock)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	224 elif args[0] == 'run_default':
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	225 mc = MnistCrbm(DEFAULT_STATE)
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	226 mc.train()
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	227
ffbf0e41bcee Aded code to run experiment on cluster, separate configuration from other machinery. Not tested yet. fsavard parents: 337 diff changeset	228

Mercurial > ift6266

annotate deep/crbm/mnist_crbm.py @ 343:82dae7c46046