pylearn: learner.py annotate

annotate learner.py @ 194:0040ba0f0666

small refactoring

author	Frederic Bastien <bastienf@iro.umontreal.ca>
date	Tue, 13 May 2008 15:11:24 -0400
parents	fb4837eed1a6
children	69759976b3ac

rev	line source
1 2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	1
172 fb4837eed1a6 fixed import of AbstractFunction Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 167 diff changeset	2 from exceptions import *
fb4837eed1a6 fixed import of AbstractFunction Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 167 diff changeset	3 from dataset import AttributesHolder,ApplyFunctionDataSet,DataSet,CachedDataSet
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	4 import theano
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	5 from theano import compile
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	6 from theano import tensor as t
1 2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	7
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	8 class Learner(AttributesHolder):
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	9 """
f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	10 Base class for learning algorithms, provides an interface
1 2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	11 that allows various algorithms to be applicable to generic learning
2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	12 algorithms.
2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	13
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	14 A L{Learner} can be seen as a learning algorithm, a function that when
135 0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	15 applied to training data returns a learned function (which is an object that
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	16 can be applied to other data and return some output data).
1 2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	17 """
2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	18
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	19 def __init__(self):
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	20 pass
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	21
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	22 def forget(self):
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	23 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	24 Reset the state of the learner to a blank slate, before seeing
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	25 training data. The operation may be non-deterministic if the
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	26 learner has a random number generator that is set to use a
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	27 different seed each time it forget() is called.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	28 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	29 raise NotImplementedError
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	30
14 5ede27026e05 Working on gradient_based_learner bengioy@bengiomac.local parents: 13 diff changeset	31 def update(self,training_set,train_stats_collector=None):
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	32 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	33 Continue training a learner, with the evidence provided by the given training set.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	34 Hence update can be called multiple times. This is particularly useful in the
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	35 on-line setting or the sequential (Bayesian or not) settings.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	36 The result is a function that can be applied on data, with the same
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	37 semantics of the Learner.use method.
75 90e4c0784d6e Added draft of LinearRegression learner bengioy@bengiomac.local parents: 20 diff changeset	38
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	39 The user may optionally provide a training L{StatsCollector} that is used to record
75 90e4c0784d6e Added draft of LinearRegression learner bengioy@bengiomac.local parents: 20 diff changeset	40 some statistics of the outputs computed during training. It is update(d) during
90e4c0784d6e Added draft of LinearRegression learner bengioy@bengiomac.local parents: 20 diff changeset	41 training.
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	42 """
13 633453635d51 Starting to work on gradient_based_learner.py bengioy@bengiomac.local parents: 10 diff changeset	43 return self.use # default behavior is 'non-adaptive', i.e. update does not do anything
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	44
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	45
20 266c68cb6136 Minor editions, plus adding untested ApplyFunctionDataset for GradientLearner in the works. bengioy@bengiomac.local parents: 14 diff changeset	46 def __call__(self,training_set,train_stats_collector=None):
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	47 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	48 Train a learner from scratch using the provided training set,
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	49 and return the learned function.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	50 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	51 self.forget()
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	52 return self.update(training_set,train_stats_collector)
1 2cd82666b9a7 Added statscollector and started writing dataset and learner. bengioy@esprit.iro.umontreal.ca parents: 0 diff changeset	53
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	54 def use(self,input_dataset,output_fieldnames=None,
135 0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	55 test_stats_collector=None,copy_inputs=False,
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	56 put_stats_in_output_dataset=True,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	57 output_attributes=[]):
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	58 """
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	59 Once a L{Learner} has been trained by one or more call to 'update', it can
f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	60 be used with one or more calls to 'use'. The argument is an input L{DataSet} (possibly
f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	61 containing a single example) and the result is an output L{DataSet} of the same length.
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	62 If output_fieldnames is specified, it may be use to indicate which fields should
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	63 be constructed in the output L{DataSet} (for example ['output','classification_error']).
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	64 Otherwise, self.defaultOutputFields is called to choose the output fields.
20 266c68cb6136 Minor editions, plus adding untested ApplyFunctionDataset for GradientLearner in the works. bengioy@bengiomac.local parents: 14 diff changeset	65 Optionally, if copy_inputs, the input fields (of the input_dataset) can be made
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	66 visible in the output L{DataSet} returned by this method.
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	67 Optionally, attributes of the learner can be copied in the output dataset,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	68 and statistics computed by the stats collector also put in the output dataset.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	69 Note the distinction between fields (which are example-wise quantities, e.g. 'input')
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	70 and attributes (which are not, e.g. 'regularization_term').
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	71
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	72 We provide here a default implementation that does all this using
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	73 a sub-class defined method: minibatchwiseUseFunction.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	74
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	75 @todo check if some of the learner attributes are actually SPECIFIED
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	76 as attributes of the input_dataset, and if so use their values instead
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	77 of the ones in the learner.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	78
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	79 The learner tries to compute in the output dataset the output fields specified.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	80 If None is specified then self.defaultOutputFields(input_dataset.fieldNames())
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	81 is called to determine the output fields.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	82
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	83 Attributes of the learner can also optionally be copied into the output dataset.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	84 If output_attributes is None then all of the attributes in self.AttributeNames()
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	85 are copied in the output dataset, but if it is [] (the default), then none are copied.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	86 If a test_stats_collector is provided, then its attributes (test_stats_collector.AttributeNames())
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	87 are also copied into the output dataset attributes.
10 80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners. bengioy@esprit.iro.umontreal.ca parents: 1 diff changeset	88 """
135 0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	89 input_fieldnames = input_dataset.fieldNames()
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	90 if not output_fieldnames:
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	91 output_fieldnames = self.defaultOutputFields(input_fieldnames)
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	92
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	93 minibatchwise_use_function = self.minibatchwiseUseFunction(input_fieldnames,
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	94 output_fieldnames,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	95 test_stats_collector)
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	96 virtual_output_dataset = ApplyFunctionDataSet(input_dataset,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	97 minibatchwise_use_function,
135 0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 134 diff changeset	98 output_fieldnames,
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	99 True,DataSet.numpy_vstack,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	100 DataSet.numpy_hstack)
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	101 # actually force the computation
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	102 output_dataset = CachedDataSet(virtual_output_dataset,True)
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	103 if copy_inputs:
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	104 output_dataset = input_dataset \| output_dataset
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	105 # copy the wanted attributes in the dataset
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	106 if output_attributes is None:
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	107 output_attributes = self.attributeNames()
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	108 if output_attributes:
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	109 assert set(attribute_names) <= set(self.attributeNames())
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	110 output_dataset.setAttributes(output_attributes,
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	111 self.names2attributes(output_attributes,return_copy=True))
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	112 if test_stats_collector:
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	113 test_stats_collector.update(output_dataset)
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	114 if put_stats_in_output_dataset:
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	115 output_dataset.setAttributes(test_stats_collector.attributeNames(),
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	116 test_stats_collector.attributes())
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	117 return output_dataset
77 1e2bb5bad636 toying with different ways to implement learners bengioy@bengiomac.local parents: 75 diff changeset	118
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	119 def minibatchwiseUseFunction(self, input_fields, output_fields, stats_collector):
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	120 """
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	121 Returns a function that can map the given input fields to the given output fields
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	122 and to the attributes that the stats collector needs for its computation.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	123 That function is expected to operate on minibatches.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	124 The function returned makes use of the self.useInputAttributes() and
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	125 sets the attributes specified by self.useOutputAttributes().
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	126 """
143 b7ca3545186b added missing raise Frederic Bastien <bastienf@iro.umontreal.ca> parents: 134 diff changeset	127 raise AbstractFunction()
b7ca3545186b added missing raise Frederic Bastien <bastienf@iro.umontreal.ca> parents: 134 diff changeset	128
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	129 def attributeNames(self):
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	130 """
3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	131 A Learner may have attributes that it wishes to export to other objects. To automate
3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	132 such export, sub-classes should define here the names (list of strings) of these attributes.
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	133
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	134 @todo By default, attributeNames looks for all dictionary entries whose name does not start with _.
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	135 """
3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	136 return []
77 1e2bb5bad636 toying with different ways to implement learners bengioy@bengiomac.local parents: 75 diff changeset	137
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	138 def attributes(self,return_copy=False):
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	139 """
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	140 Return a list with the values of the learner's attributes (or optionally, a deep copy).
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	141 """
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	142 return self.names2attributes(self.attributeNames(),return_copy)
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	143
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	144 def names2attributes(self,names):
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	145 """
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	146 Private helper function that maps a list of attribute names to a list
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	147 of (optionally copies) values of attributes.
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	148 """
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	149 res=[]
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	150 for name in names:
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	151 assert name in names
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	152 res.append(self.__getattribute__(name))
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	153 return res
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	154
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	155 def useInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	156 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	157 A subset of self.attributeNames() which are the names of attributes needed by use() in order
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	158 to do its work.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	159 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	160 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	161
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	162 def useOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	163 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	164 A subset of self.attributeNames() which are the names of attributes modified/created by use() in order
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	165 to do its work.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	166 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	167 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	168
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	169
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	170 class TLearner(Learner):
3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	171 """
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	172 TLearner is a virtual class of L{Learner}s that attempts to factor
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	173 out of the definition of a learner the steps that are common to
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	174 many implementations of learning algorithms, so as to leave only
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	175 'the equations' to define in particular sub-classes, using Theano.
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	176
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	177 In the default implementations of use and update, it is assumed
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	178 that the 'use' and 'update' methods visit examples in the input
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	179 dataset sequentially. In the 'use' method only one pass through the
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	180 dataset is done, whereas the sub-learner may wish to iterate over
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	181 the examples multiple times. Subclasses where this basic model is
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	182 not appropriate can simply redefine update or use.
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	183
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	184 Sub-classes must provide the following functions and functionalities:
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	185 - attributeNames(): defines all the names of attributes which can
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	186 be used as fields or
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	187 attributes in input/output datasets or in
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	188 stats collectors. All these attributes
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	189 are expected to be theano.Result objects
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	190 (with a .data property and recognized by
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	191 theano.Function for compilation). The sub-class
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	192 constructor defines the relations between the
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	193 Theano variables that may be used by 'use'
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	194 and 'update' or by a stats collector.
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	195 - defaultOutputFields(input_fields): return a list of default
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	196 dataset output fields when
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	197 None are provided by the caller of use.
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	198 The following naming convention is assumed and important. Attributes
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	199 whose names are listed in attributeNames() can be of any type,
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	200 but those that can be referenced as input/output dataset fields or
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	201 as output attributes in 'use' or as input attributes in the stats
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	202 collector should be associated with a Theano Result variable. If the
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	203 exported attribute name is <name>, the corresponding Result name
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	204 (an internal attribute of the TLearner, created in the sub-class
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	205 constructor) should be _<name>. Typically <name> will be numpy
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	206 ndarray and _<name> will be the corresponding Theano Tensor (for
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	207 symbolic manipulation).
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	208
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	209 @todo pousser dans Learner toute la poutine qui peut l'etre sans etre
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	210 dependant de Theano
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	211 """
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	212
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	213 def __init__(self):
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	214 Learner.__init__(self)
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	215 self.use_functions_dictionary={}
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	216
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	217 def defaultOutputFields(self, input_fields):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	218 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	219 Return a default list of output field names (to put in the output dataset).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	220 This will be used when None are provided (as output_fields) by the caller of the 'use' method.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	221 This may involve looking at the input_fields (names) available in the
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	222 input_dataset.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	223 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	224 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	225
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	226 def minibatchwiseUseFunction(self, input_fields, output_fields, stats_collector):
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	227 """
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	228 Implement minibatchwiseUseFunction by exploiting Theano compilation
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	229 and the expression graph defined by a sub-class constructor.
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	230 """
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	231 if stats_collector:
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	232 stats_collector_inputs = stats_collector.input2UpdateAttributes()
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	233 for attribute in stats_collector_inputs:
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	234 if attribute not in input_fields:
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	235 output_fields.append(attribute)
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	236 key = (tuple(input_fields),tuple(output_fields))
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	237 if key not in self.use_functions_dictionary:
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	238 use_input_attributes = self.useInputAttributes()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	239 use_output_attributes = self.useOutputAttributes()
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	240 complete_f = compile.function(self.names2OpResults(input_fields+use_input_attributes),
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	241 self.names2OpResults(output_fields+use_output_attributes))
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	242 def f(*input_field_values):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	243 input_attribute_values = self.names2attributes(use_input_attributes)
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	244 results = complete_f(*(list(input_field_values) + input_attribute_values))
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	245 output_field_values = results[0:len(output_fields)]
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	246 output_attribute_values = results[len(output_fields):len(results)]
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	247 if use_output_attributes:
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	248 self.setAttributes(use_output_attributes,output_attribute_values)
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	249 return output_field_values
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	250 self.use_functions_dictionary[key]=f
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	251 return self.use_functions_dictionary[key]
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	252
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	253 def names2OpResults(self,names):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	254 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	255 Private helper function that maps a list of attribute names to a list
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	256 of corresponding Op Results (with the same name but with a '_' prefix).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	257 """
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	258 return [self.__getattribute__('_'+name) for name in names]
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	259
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	260
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	261 class MinibatchUpdatesTLearner(TLearner):
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	262 """
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	263 This adds the following functions to a L{TLearner}:
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	264 - updateStart(), updateEnd(), updateMinibatch(minibatch), isLastEpoch():
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	265 functions executed at the beginning, the end, in the middle (for
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	266 each minibatch) of the update method, and at the end of each
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	267 epoch. This model only works for 'online' or one-shot learning
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	268 that requires going only once through the training data. For more
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	269 complicated models, more specialized subclasses of TLearner should
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	270 be used or a learning-algorithm specific update method should
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	271 be defined.
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	272
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	273 - a 'parameters' attribute which is a list of parameters
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	274 (whose names are specified by the user's subclass with the
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	275 parameterAttributes() method)
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 144 diff changeset	276
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	277 """
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	278
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	279 def __init__(self):
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	280 TLearner.__init__(self)
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	281 self.update_minibatch_function = compile.function(self.names2OpResults(self.updateMinibatchOutputAttributes()+
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	282 self.updateMinibatchInputFields()),
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	283 self.names2OpResults(self.updateMinibatchOutputAttributes()))
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	284 self.update_end_function = compile.function(self.names2OpResults(self.updateEndInputAttributes()),
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	285 self.names2OpResults(self.updateEndOutputAttributes()))
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	286
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	287 def allocate(self, minibatch):
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	288 """
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	289 This function is called at the beginning of each L{updateMinibatch}
128 ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	290 and should be used to check that all required attributes have been
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	291 allocated and initialized (usually this function calls forget()
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	292 when it has to do an initialization).
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	293 """
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	294 raise AbstractFunction()
ee5507af2c60 minor edits Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 126 diff changeset	295
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	296 def updateMinibatchInputFields(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	297 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	298
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	299 def updateMinibatchInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	300 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	301
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	302 def updateMinibatchOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	303 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	304
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	305 def updateEndInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	306 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	307
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	308 def updateEndOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	309 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	310
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	311 def parameterAttributes(self):
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	312 raise AbstractFunction()
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	313
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	314 def updateStart(self,training_set):
b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	315 pass
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	316
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	317 def updateEnd(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	318 self.setAttributes(self.updateEndOutputAttributes(),
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	319 self.update_end_function(*self.names2attributes(self.updateEndInputAttributes())))
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	320 self.parameters = self.names2attributes(self.parameterAttributes())
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	321
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	322 def updateMinibatch(self,minibatch):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	323 # make sure all required fields are allocated and initialized
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	324 self.allocate(minibatch)
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	325 input_attributes = self.names2attributes(self.updateMinibatchInputAttributes())
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	326 input_fields = minibatch(*self.updateMinibatchInputFields())
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	327 self.setAttributes(self.updateMinibatchOutputAttributes(),
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	328 # concatenate the attribute values and field values and then apply update fn
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	329 self.update_minibatch_function(*(input_attributes+input_fields)))
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	330
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	331 def isLastEpoch(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	332 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	333 This method is called at the end of each epoch (cycling over the training set).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	334 It returns a boolean to indicate if this is the last epoch.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	335 By default just do one epoch.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	336 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	337 return True
78 3499918faa9d In the middle of designing TLearner bengioy@bengiomac.local parents: 77 diff changeset	338
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	339 def update(self,training_set,train_stats_collector=None):
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	340 """
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	341 @todo check if some of the learner attributes are actually SPECIFIED
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	342 in as attributes of the training_set.
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	343 """
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	344 self.updateStart(training_set)
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	345 stop=False
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	346 if hasattr(self,'_minibatch_size') and self._minibatch_size:
b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	347 minibatch_size=self._minibatch_size
b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	348 else:
b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	349 minibatch_size=min(100,len(training_set))
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	350 while not stop:
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	351 if train_stats_collector:
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	352 train_stats_collector.forget() # restart stats collectin at the beginning of each epoch
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	353 for minibatch in training_set.minibatches(minibatch_size=minibatch_size):
b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	354 self.updateMinibatch(minibatch)
107 c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	355 if train_stats_collector:
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	356 minibatch_set = minibatch.examples()
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	357 minibatch_set.setAttributes(self.attributeNames(),self.attributes())
c4916445e025 Comments from Pascal V. Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 92 diff changeset	358 train_stats_collector.update(minibatch_set)
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	359 stop = self.isLastEpoch()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 109 diff changeset	360 self.updateEnd()
92 c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	361 return self.use
c4726e19b8ec Finished first draft of TLearner Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 78 diff changeset	362
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	363 class OnlineGradientTLearner(MinibatchUpdatesTLearner):
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	364 """
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	365 Specialization of L{MinibatchUpdatesTLearner} in which the minibatch updates
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	366 are obtained by performing an online (minibatch-based) gradient step.
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	367
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	368 Sub-classes must define the following:
132 f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	369 - self._learning_rate (may be changed by the sub-class between epochs or minibatches)
f6505ec32dc3 Updated documentation slightly Joseph Turian <turian@gmail.com> parents: 131 diff changeset	370 - self.lossAttribute() = name of the loss field
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	371 """
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	372 def __init__(self,truly_online=False):
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	373 """
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	374 If truly_online then only one pass is made through the training set passed to update().
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	375
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	376 SUBCLASSES SHOULD CALL THIS CONSTRUCTOR ONLY AFTER HAVING DEFINED ALL THEIR THEANO FORMULAS
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	377 """
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	378 self.truly_online=truly_online
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	379
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	380 # create the formulas for the gradient update
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	381 old_params = [self.__getattribute__("_"+name) for name in self.parameterAttributes()]
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	382 new_params_names = ["_new_"+name for name in self.parameterAttributes()]
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	383 loss = self.__getattribute__("_"+self.lossAttribute())
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	384 self.setAttributes(new_params_names,
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	385 [t.add_inplace(param,self._learning_rate*t.grad(loss,param))
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	386 for param in old_params])
129 4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	387 MinibatchUpdatesTLearner.__init__(self)
4c2280edcaf5 Fixed typos in learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 128 diff changeset	388
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	389
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	390 def namesOfAttributesToComputeOutputs(self,output_names):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	391 """
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	392 The output_names are attribute names (not the corresponding Result names, which have leading _).
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	393 Return the corresponding input names
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	394 """
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	395 all_inputs = t.gof.graph.inputs(self.names2OpResults(output_names))
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	396 # remove constants and leading '_' in name
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	397
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	398 return [r.name for r in all_inputs if isinstance(r,theano.Result) and \
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	399 not isinstance(r,theano.Constant) and not isinstance(r,theano.Value)]
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	400 #inputs = []
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	401 #for r in all_inputs:
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	402 # if isinstance(r,theano.Result) and \
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	403 # not isinstance(r,theano.Constant) and not isinstance(r,theano.Value):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	404 # inputs.append(r.name)
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	405 #return inputs
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	406
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	407 def isLastEpoch(self):
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	408 return self.truly_online
88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	409
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	410 def updateMinibatchInputAttributes(self):
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	411 return self.parameterAttributes()
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	412
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	413 def updateMinibatchOutputAttributes(self):
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 132 diff changeset	414 return ["new_"+name for name in self.parameterAttributes()]
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	415
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	416 def updateEndInputAttributes(self):
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	417 return self.namesOfAttributesToComputeOutputs(self.updateEndOutputAttributes())
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	418
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	419 def useInputAttributes(self):
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	420 return self.parameterAttributes()
111 88257dfedf8c Added another work in progress, for mlp's bengioy@bengiomac.local parents: 110 diff changeset	421
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	422 def useOutputAttributes(self):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	423 return []
118 d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 111 diff changeset	424

Mercurial > pylearn

annotate learner.py @ 194:0040ba0f0666