annotate learner.py @ 204:62c7527c9ec1

debugging mlp.py
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Wed, 14 May 2008 14:06:15 -0400
parents 2698c0feeb54
children cb6b945acf5a
rev   line source
1
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
1
172
fb4837eed1a6 fixed import of AbstractFunction
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 167
diff changeset
2 from exceptions import *
fb4837eed1a6 fixed import of AbstractFunction
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 167
diff changeset
3 from dataset import AttributesHolder,ApplyFunctionDataSet,DataSet,CachedDataSet
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
4 import theano
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
5 from theano import compile
129
4c2280edcaf5 Fixed typos in learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 128
diff changeset
6 from theano import tensor as t
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
7 from misc import Print
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
8 Print = lambda x: lambda y: y
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
9
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
10 class Learner(AttributesHolder):
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
11 """
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
12 Base class for learning algorithms, provides an interface
1
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
13 that allows various algorithms to be applicable to generic learning
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
14 algorithms.
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
15
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
16 A L{Learner} can be seen as a learning algorithm, a function that when
135
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
17 applied to training data returns a learned function (which is an object that
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
18 can be applied to other data and return some output data).
1
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
19 """
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
20
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
21 def __init__(self):
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
22 pass
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
23
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
24 def forget(self):
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
25 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
26 Reset the state of the learner to a blank slate, before seeing
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
27 training data. The operation may be non-deterministic if the
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
28 learner has a random number generator that is set to use a
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
29 different seed each time it forget() is called.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
30 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
31 raise NotImplementedError
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
32
14
5ede27026e05 Working on gradient_based_learner
bengioy@bengiomac.local
parents: 13
diff changeset
33 def update(self,training_set,train_stats_collector=None):
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
34 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
35 Continue training a learner, with the evidence provided by the given training set.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
36 Hence update can be called multiple times. This is particularly useful in the
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
37 on-line setting or the sequential (Bayesian or not) settings.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
38 The result is a function that can be applied on data, with the same
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
39 semantics of the Learner.use method.
75
90e4c0784d6e Added draft of LinearRegression learner
bengioy@bengiomac.local
parents: 20
diff changeset
40
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
41 The user may optionally provide a training L{StatsCollector} that is used to record
75
90e4c0784d6e Added draft of LinearRegression learner
bengioy@bengiomac.local
parents: 20
diff changeset
42 some statistics of the outputs computed during training. It is update(d) during
90e4c0784d6e Added draft of LinearRegression learner
bengioy@bengiomac.local
parents: 20
diff changeset
43 training.
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
44 """
13
633453635d51 Starting to work on gradient_based_learner.py
bengioy@bengiomac.local
parents: 10
diff changeset
45 return self.use # default behavior is 'non-adaptive', i.e. update does not do anything
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
46
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
47
20
266c68cb6136 Minor editions, plus adding untested ApplyFunctionDataset for GradientLearner in the works.
bengioy@bengiomac.local
parents: 14
diff changeset
48 def __call__(self,training_set,train_stats_collector=None):
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
49 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
50 Train a learner from scratch using the provided training set,
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
51 and return the learned function.
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
52 """
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
53 self.forget()
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
54 return self.update(training_set,train_stats_collector)
1
2cd82666b9a7 Added statscollector and started writing dataset and learner.
bengioy@esprit.iro.umontreal.ca
parents: 0
diff changeset
55
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
56 def use(self,input_dataset,output_fieldnames=None,
135
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
57 test_stats_collector=None,copy_inputs=False,
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
58 put_stats_in_output_dataset=True,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
59 output_attributes=[]):
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
60 """
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
61 Once a L{Learner} has been trained by one or more call to 'update', it can
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
62 be used with one or more calls to 'use'. The argument is an input L{DataSet} (possibly
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
63 containing a single example) and the result is an output L{DataSet} of the same length.
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
64 If output_fieldnames is specified, it may be use to indicate which fields should
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
65 be constructed in the output L{DataSet} (for example ['output','classification_error']).
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
66 Otherwise, self.defaultOutputFields is called to choose the output fields.
20
266c68cb6136 Minor editions, plus adding untested ApplyFunctionDataset for GradientLearner in the works.
bengioy@bengiomac.local
parents: 14
diff changeset
67 Optionally, if copy_inputs, the input fields (of the input_dataset) can be made
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
68 visible in the output L{DataSet} returned by this method.
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
69 Optionally, attributes of the learner can be copied in the output dataset,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
70 and statistics computed by the stats collector also put in the output dataset.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
71 Note the distinction between fields (which are example-wise quantities, e.g. 'input')
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
72 and attributes (which are not, e.g. 'regularization_term').
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
73
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
74 We provide here a default implementation that does all this using
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
75 a sub-class defined method: minibatchwiseUseFunction.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
76
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
77 @todo check if some of the learner attributes are actually SPECIFIED
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
78 as attributes of the input_dataset, and if so use their values instead
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
79 of the ones in the learner.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
80
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
81 The learner tries to compute in the output dataset the output fields specified.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
82 If None is specified then self.defaultOutputFields(input_dataset.fieldNames())
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
83 is called to determine the output fields.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
84
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
85 Attributes of the learner can also optionally be copied into the output dataset.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
86 If output_attributes is None then all of the attributes in self.AttributeNames()
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
87 are copied in the output dataset, but if it is [] (the default), then none are copied.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
88 If a test_stats_collector is provided, then its attributes (test_stats_collector.AttributeNames())
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
89 are also copied into the output dataset attributes.
10
80bf5492e571 Rewrote learner.py according to the specs in the wiki for learners.
bengioy@esprit.iro.umontreal.ca
parents: 1
diff changeset
90 """
135
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
91 input_fieldnames = input_dataset.fieldNames()
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
92 if not output_fieldnames:
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
93 output_fieldnames = self.defaultOutputFields(input_fieldnames)
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
94
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
95 minibatchwise_use_function = self.minibatchwiseUseFunction(input_fieldnames,
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
96 output_fieldnames,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
97 test_stats_collector)
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
98 virtual_output_dataset = ApplyFunctionDataSet(input_dataset,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
99 minibatchwise_use_function,
135
0d8e721cc63c Fixed bugs in dataset to make test_mlp.py work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 134
diff changeset
100 output_fieldnames,
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
101 True,DataSet.numpy_vstack,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
102 DataSet.numpy_hstack)
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
103 # actually force the computation
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
104 output_dataset = CachedDataSet(virtual_output_dataset,True)
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
105 if copy_inputs:
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
106 output_dataset = input_dataset | output_dataset
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
107 # copy the wanted attributes in the dataset
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
108 if output_attributes is None:
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
109 output_attributes = self.attributeNames()
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
110 if output_attributes:
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
111 assert set(attribute_names) <= set(self.attributeNames())
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
112 output_dataset.setAttributes(output_attributes,
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
113 self.names2attributes(output_attributes,return_copy=True))
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
114 if test_stats_collector:
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
115 test_stats_collector.update(output_dataset)
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
116 if put_stats_in_output_dataset:
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
117 output_dataset.setAttributes(test_stats_collector.attributeNames(),
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
118 test_stats_collector.attributes())
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
119 return output_dataset
77
1e2bb5bad636 toying with different ways to implement learners
bengioy@bengiomac.local
parents: 75
diff changeset
120
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
121 def minibatchwiseUseFunction(self, input_fields, output_fields, stats_collector):
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
122 """
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
123 Returns a function that can map the given input fields to the given output fields
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
124 and to the attributes that the stats collector needs for its computation.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
125 That function is expected to operate on minibatches.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
126 The function returned makes use of the self.useInputAttributes() and
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
127 sets the attributes specified by self.useOutputAttributes().
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
128 """
143
b7ca3545186b added missing raise
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 134
diff changeset
129 raise AbstractFunction()
b7ca3545186b added missing raise
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 134
diff changeset
130
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
131 def attributeNames(self):
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
132 """
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
133 A Learner may have attributes that it wishes to export to other objects. To automate
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
134 such export, sub-classes should define here the names (list of strings) of these attributes.
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
135
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
136 @todo By default, attributeNames looks for all dictionary entries whose name does not start with _.
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
137 """
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
138 return []
77
1e2bb5bad636 toying with different ways to implement learners
bengioy@bengiomac.local
parents: 75
diff changeset
139
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
140 def attributes(self,return_copy=False):
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
141 """
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
142 Return a list with the values of the learner's attributes (or optionally, a deep copy).
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
143 """
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
144 return self.names2attributes(self.attributeNames(),return_copy)
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
145
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
146 def names2attributes(self,names):
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
147 """
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
148 Private helper function that maps a list of attribute names to a list
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
149 of (optionally copies) values of attributes.
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
150 """
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
151 res=[]
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
152 for name in names:
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
153 assert name in names
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
154 res.append(self.__getattribute__(name))
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
155 return res
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
156
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
157 def useInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
158 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
159 A subset of self.attributeNames() which are the names of attributes needed by use() in order
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
160 to do its work.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
161 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
162 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
163
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
164 def useOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
165 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
166 A subset of self.attributeNames() which are the names of attributes modified/created by use() in order
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
167 to do its work.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
168 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
169 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
170
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
171
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
172 class TLearner(Learner):
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
173 """
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
174 TLearner is a virtual class of L{Learner}s that attempts to factor
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
175 out of the definition of a learner the steps that are common to
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
176 many implementations of learning algorithms, so as to leave only
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
177 'the equations' to define in particular sub-classes, using Theano.
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
178
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
179 In the default implementations of use and update, it is assumed
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
180 that the 'use' and 'update' methods visit examples in the input
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
181 dataset sequentially. In the 'use' method only one pass through the
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
182 dataset is done, whereas the sub-learner may wish to iterate over
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
183 the examples multiple times. Subclasses where this basic model is
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
184 not appropriate can simply redefine update or use.
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
185
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
186 Sub-classes must provide the following functions and functionalities:
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
187 - attributeNames(): defines all the names of attributes which can
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
188 be used as fields or
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
189 attributes in input/output datasets or in
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
190 stats collectors. All these attributes
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
191 are expected to be theano.Result objects
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
192 (with a .data property and recognized by
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
193 theano.function for compilation). The sub-class
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
194 constructor defines the relations between the
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
195 Theano variables that may be used by 'use'
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
196 and 'update' or by a stats collector.
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
197 - defaultOutputFields(input_fields): return a list of default
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
198 dataset output fields when
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
199 None are provided by the caller of use.
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
200 The following naming convention is assumed and important. Attributes
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
201 whose names are listed in attributeNames() can be of any type,
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
202 but those that can be referenced as input/output dataset fields or
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
203 as output attributes in 'use' or as input attributes in the stats
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
204 collector should be associated with a Theano Result variable. If the
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
205 exported attribute name is <name>, the corresponding Result name
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
206 (an internal attribute of the TLearner, created in the sub-class
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
207 constructor) should be _<name>. Typically <name> will be numpy
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
208 ndarray and _<name> will be the corresponding Theano Tensor (for
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
209 symbolic manipulation).
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
210
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
211 @todo pousser dans Learner toute la poutine qui peut l'etre sans etre
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
212 dependant de Theano
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
213 """
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
214
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
215 def __init__(self,linker="c|py"):
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
216 Learner.__init__(self)
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
217 self.use_functions_dictionary={}
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
218 self.linker=linker
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
219
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
220 def defaultOutputFields(self, input_fields):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
221 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
222 Return a default list of output field names (to put in the output dataset).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
223 This will be used when None are provided (as output_fields) by the caller of the 'use' method.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
224 This may involve looking at the input_fields (names) available in the
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
225 input_dataset.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
226 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
227 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
228
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
229 def minibatchwiseUseFunction(self, input_fields, output_fields, stats_collector):
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
230 """
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
231 Implement minibatchwiseUseFunction by exploiting Theano compilation
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
232 and the expression graph defined by a sub-class constructor.
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
233 """
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
234 if stats_collector:
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
235 stats_collector_inputs = stats_collector.input2UpdateAttributes()
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
236 for attribute in stats_collector_inputs:
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
237 if attribute not in input_fields:
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
238 output_fields.append(attribute)
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
239 key = (tuple(input_fields),tuple(output_fields))
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
240 if key not in self.use_functions_dictionary:
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
241 use_input_attributes = self.useInputAttributes()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
242 use_output_attributes = self.useOutputAttributes()
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
243 complete_f = compile.function(self.names2OpResults(input_fields+use_input_attributes),
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
244 self.names2OpResults(output_fields+use_output_attributes),
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
245 self.linker)
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
246 def f(*input_field_values):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
247 input_attribute_values = self.names2attributes(use_input_attributes)
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
248 results = complete_f(*(list(input_field_values) + input_attribute_values))
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
249 output_field_values = results[0:len(output_fields)]
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
250 output_attribute_values = results[len(output_fields):len(results)]
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
251 if use_output_attributes:
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
252 self.setAttributes(use_output_attributes,output_attribute_values)
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
253 return output_field_values
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
254 self.use_functions_dictionary[key]=f
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
255 return self.use_functions_dictionary[key]
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
256
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
257 def names2OpResults(self,names):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
258 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
259 Private helper function that maps a list of attribute names to a list
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
260 of corresponding Op Results (with the same name but with a '_' prefix).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
261 """
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
262 return [self.__getattribute__('_'+name) for name in names]
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
263
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
264
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
265 class MinibatchUpdatesTLearner(TLearner):
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
266 """
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
267 This adds the following functions to a L{TLearner}:
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
268 - updateStart(), updateEnd(), updateMinibatch(minibatch), isLastEpoch():
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
269 functions executed at the beginning, the end, in the middle (for
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
270 each minibatch) of the update method, and at the end of each
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
271 epoch. This model only works for 'online' or one-shot learning
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
272 that requires going only once through the training data. For more
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
273 complicated models, more specialized subclasses of TLearner should
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
274 be used or a learning-algorithm specific update method should
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
275 be defined.
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
276
167
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
277 - a 'parameters' attribute which is a list of parameters
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
278 (whose names are specified by the user's subclass with the
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
279 parameterAttributes() method)
4803cb76e26b Updated documentation
Joseph Turian <turian@gmail.com>
parents: 144
diff changeset
280
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
281 """
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
282
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
283 def __init__(self,linker="c|py"):
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
284 TLearner.__init__(self,linker)
177
69759976b3ac s/Output/Input
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 172
diff changeset
285 self.update_minibatch_function = compile.function(self.names2OpResults(self.updateMinibatchInputAttributes()+
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
286 self.updateMinibatchInputFields()),
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
287 self.names2OpResults(self.updateMinibatchOutputAttributes()),
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
288 linker)
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
289 self.update_end_function = compile.function(self.names2OpResults(self.updateEndInputAttributes()),
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
290 self.names2OpResults(self.updateEndOutputAttributes()),
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
291 linker)
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
292
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
293 def allocate(self, minibatch):
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
294 """
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
295 This function is called at the beginning of each L{updateMinibatch}
128
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
296 and should be used to check that all required attributes have been
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
297 allocated and initialized (usually this function calls forget()
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
298 when it has to do an initialization).
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
299 """
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
300 raise AbstractFunction()
ee5507af2c60 minor edits
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 126
diff changeset
301
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
302 def updateMinibatchInputFields(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
303 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
304
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
305 def updateMinibatchInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
306 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
307
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
308 def updateMinibatchOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
309 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
310
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
311 def updateEndInputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
312 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
313
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
314 def updateEndOutputAttributes(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
315 raise AbstractFunction()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
316
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
317 def parameterAttributes(self):
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
318 raise AbstractFunction()
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
319
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
320 def updateStart(self,training_set):
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
321 pass
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
322
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
323 def updateEnd(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
324 self.setAttributes(self.updateEndOutputAttributes(),
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
325 self.update_end_function(*self.names2attributes(self.updateEndInputAttributes())))
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
326 self.parameters = self.names2attributes(self.parameterAttributes())
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
327
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
328 def updateMinibatch(self,minibatch):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
329 # make sure all required fields are allocated and initialized
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
330 self.allocate(minibatch)
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
331 input_attributes = self.names2attributes(self.updateMinibatchInputAttributes())
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
332 input_fields = minibatch(*self.updateMinibatchInputFields())
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
333 self.setAttributes(self.updateMinibatchOutputAttributes(),
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
334 # concatenate the attribute values and field values and then apply update fn
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
335 self.update_minibatch_function(*(input_attributes+input_fields)))
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
336
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
337 def isLastEpoch(self):
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
338 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
339 This method is called at the end of each epoch (cycling over the training set).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
340 It returns a boolean to indicate if this is the last epoch.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
341 By default just do one epoch.
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
342 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
343 return True
78
3499918faa9d In the middle of designing TLearner
bengioy@bengiomac.local
parents: 77
diff changeset
344
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
345 def update(self,training_set,train_stats_collector=None):
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
346 """
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
347 @todo check if some of the learner attributes are actually SPECIFIED
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
348 in as attributes of the training_set.
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
349 """
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
350 self.updateStart(training_set)
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
351 stop=False
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
352 if hasattr(self,'_minibatch_size') and self._minibatch_size:
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
353 minibatch_size=self._minibatch_size
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
354 else:
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
355 minibatch_size=min(100,len(training_set))
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
356 while not stop:
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
357 if train_stats_collector:
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
358 train_stats_collector.forget() # restart stats collectin at the beginning of each epoch
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
359 for minibatch in training_set.minibatches(minibatch_size=minibatch_size):
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
360 self.updateMinibatch(minibatch)
107
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
361 if train_stats_collector:
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
362 minibatch_set = minibatch.examples()
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
363 minibatch_set.setAttributes(self.attributeNames(),self.attributes())
c4916445e025 Comments from Pascal V.
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 92
diff changeset
364 train_stats_collector.update(minibatch_set)
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
365 stop = self.isLastEpoch()
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 109
diff changeset
366 self.updateEnd()
92
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
367 return self.use
c4726e19b8ec Finished first draft of TLearner
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 78
diff changeset
368
129
4c2280edcaf5 Fixed typos in learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 128
diff changeset
369 class OnlineGradientTLearner(MinibatchUpdatesTLearner):
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
370 """
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
371 Specialization of L{MinibatchUpdatesTLearner} in which the minibatch updates
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
372 are obtained by performing an online (minibatch-based) gradient step.
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
373
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
374 Sub-classes must define the following:
132
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
375 - self._learning_rate (may be changed by the sub-class between epochs or minibatches)
f6505ec32dc3 Updated documentation slightly
Joseph Turian <turian@gmail.com>
parents: 131
diff changeset
376 - self.lossAttribute() = name of the loss field
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
377 """
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
378 def __init__(self,truly_online=False,linker="c|py"):
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
379 """
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
380 If truly_online then only one pass is made through the training set passed to update().
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
381
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
382 SUBCLASSES SHOULD CALL THIS CONSTRUCTOR ONLY AFTER HAVING DEFINED ALL THEIR THEANO FORMULAS
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
383 """
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
384 self.truly_online=truly_online
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
385
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
386 # create the formulas for the gradient update
129
4c2280edcaf5 Fixed typos in learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 128
diff changeset
387 old_params = [self.__getattribute__("_"+name) for name in self.parameterAttributes()]
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
388 new_params_names = ["_new_"+name for name in self.parameterAttributes()]
129
4c2280edcaf5 Fixed typos in learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 128
diff changeset
389 loss = self.__getattribute__("_"+self.lossAttribute())
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
390 self.setAttributes(new_params_names,
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
391 [t.add_inplace(param,-self._learning_rate*Print("grad("+param.name+")")(t.grad(loss,param)))
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
392 for param in old_params])
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
393 MinibatchUpdatesTLearner.__init__(self,linker)
129
4c2280edcaf5 Fixed typos in learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 128
diff changeset
394
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
395
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
396 def namesOfAttributesToComputeOutputs(self,output_names):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
397 """
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
398 The output_names are attribute names (not the corresponding Result names, which have leading _).
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
399 Return the corresponding input names
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
400 """
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
401 all_inputs = t.gof.graph.inputs(self.names2OpResults(output_names))
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
402 # remove constants and leading '_' in name
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
403
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
404 return [r.name for r in all_inputs if isinstance(r,theano.Result) and \
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
405 not isinstance(r,theano.Constant) and not isinstance(r,theano.Value)]
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
406 #inputs = []
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
407 #for r in all_inputs:
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
408 # if isinstance(r,theano.Result) and \
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
409 # not isinstance(r,theano.Constant) and not isinstance(r,theano.Value):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
410 # inputs.append(r.name)
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
411 #return inputs
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
412
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
413 def isLastEpoch(self):
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
414 return self.truly_online
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
415
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
416 def updateMinibatchInputAttributes(self):
180
2698c0feeb54 mlp seems to work!
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 177
diff changeset
417 return self.parameterAttributes()+["learning_rate"]
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
418
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
419 def updateMinibatchOutputAttributes(self):
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 132
diff changeset
420 return ["new_"+name for name in self.parameterAttributes()]
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
421
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
422 def updateEndInputAttributes(self):
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
423 return self.namesOfAttributesToComputeOutputs(self.updateEndOutputAttributes())
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
424
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
425 def useInputAttributes(self):
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
426 return self.parameterAttributes()
111
88257dfedf8c Added another work in progress, for mlp's
bengioy@bengiomac.local
parents: 110
diff changeset
427
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
428 def useOutputAttributes(self):
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
429 return []
118
d0a1bd0378c6 Finished draft of OneHiddenLayerNNetClassifier to debut learner.py
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 111
diff changeset
430