Mercurial > pylearn

annotate doc/v2_planning/plugin_RP.py @ 1201:46527ae6db53

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
architecture: Clarified what I meant about saving the model
author Olivier Delalleau <delallea@iro>
date Mon, 20 Sep 2010 17:05:15 -0400
parents fe6c25eb1e37
children 7fff3d5c7694
rev   line source
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
1 '''
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
2 =================================================
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
3 Plugin system for interative algortithm Version B
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
4 =================================================
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
5
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
6 After the meeting (September 16) we sort of stumbled on
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
7 two possible versions of the plug-in system. This represents
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
8 the second version. It suffered a few changes after seeing
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
9 Olivier's code and talking to him.
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
10
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
11 Concept
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
12 =======
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
13
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
14 The basic idea behind this version is not to have a list of all
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
15 possible events, but rather have plugin register to events.By
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
16 specifying what plugin listens to which event produced by what
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
17 plugin you define a sort of dependency graph. Structuring things
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
18 in such a graph might make the script more intuitive when reading.
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
19
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
20 I will first go through pseudo-code for two example and then enumerate
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
21 my insights and concepts on the matter
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
22
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
23
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
24 Example : Producer - Consumer that Guillaume described
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
25 ======================================================
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
26
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
27
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
28 .. code-block::
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
29 '''
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
30 sch = Schedular()
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
31
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
32 @FnPlugin(sch)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
33 def producer(self,event):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
34 self.fire('stuff', value = 'some text')
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
35
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
36 @FnPlugin(sch)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
37 def consumer(self,event):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
38 print event.value
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
39
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
40 @FnPlugin(sch)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
41 def prod_consumer(self,event):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
42 print event.value
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
43 self.fire('stuff2', value = 'stuff')
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
44
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
45 producer.act( on = Event('begin'), when = once() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
46 producer.act( on = Event('stuff'), when = always() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
47 consumer.act( on = Event('stuff'), when = always() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
48 prod_consumer.act( on = Event('stuff'), when = always() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
49
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
50 sch.run()
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
51
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
52
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
53
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
54 '''
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
55 Example : Logistic regression
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
56 =============================
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
57
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
58 Task description
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
59 ----------------
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
60
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
61 Apply a logistic regression network to some dataset. Use early stopping.
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
62 Save the weights everytime a new best score is obtained. Print trainnig score
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
63 after each epoch.
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
64
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
65
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
66 Possible script
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
67 ---------------
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
68
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
69 Notes : This would look the same for any other architecture that does not
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
70 imply pre-training ( i.e. deep networks). For example the mlp.
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
71
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
72 .. code-block::
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
73 '''
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
74
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
75 sched = Schedular()
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
76
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
77 # Data / Model Building :
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
78 # I skiped over how to design this part
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
79 # though I have some ideas
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
80 real_train_data, real_valid_data = load_mnist()
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
81 model = logreg()
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
82
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
83 # Main Plugins ( already provided in the library );
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
84 # This wrappers also registers the plugin
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
85 train_data = create_data_plugin( sched, data = real_train_data)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
86 train_model = create_train_model(sched, model = model)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
87 validate_model = create_valid_model(sched, model = model, data = valid_data)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
88 early_stopper = create_early_stopper(sched)
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
89
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
90
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
91 # On the fly plugins ( print random stuff); the main difference from my
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
92 # FnPlugin from Olivier's version is that it also register the plugin in sched
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
93 @FnPlugin(sched)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
94 def print_error(self, event):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
95 if event.type == Event('begin'):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
96 self.value = []
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
97 elif event.type == train_model.error():
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
98 self.value += [event.value]
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
99 else event.type == train_data.eod():
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
100 print 'Error :', numpy.mean(self.value)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
101
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
102 @FnPlugin(sched)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
103 def save_model(self, event):
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
104 if event.type == early_stopper.new_best_error():
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
105 cPickle.dump(model.parameters(), open('best_params.pkl','wb'))
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
106
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
107
1173
a0f178bc9052 changes during the meeting
pascanur
parents: 1172
diff changeset
108 # Create the dependency graph describing what does what
a0f178bc9052 changes during the meeting
pascanur
parents: 1172
diff changeset
109 train_data.act( on = sched.begin(), when = once() )
a0f178bc9052 changes during the meeting
pascanur
parents: 1172
diff changeset
110 train_data.act( on = Event('batch'),
a0f178bc9052 changes during the meeting
pascanur
parents: 1172
diff changeset
111 train_data.act( on = train_model.done(), when = always())
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
112 train_model.act(on = train_data.batch(), when = always())
1173
a0f178bc9052 changes during the meeting
pascanur
parents: 1172
diff changeset
113 validate_model.act(on = train_model.done(), when = every(n=10000))
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
114 early_stopper.act(on = validate_model.error(), when = always())
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
115 print_error.act( on = train_model.error(), when = always() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
116 print_error.act( on = train_data.eod(), when = always() )
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
117 save_model.act( on = eraly_stopper.new_best_errot(), when = always() )
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
118
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
119 # Run the entire thing
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
120 sched.run()
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
121
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
122
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
123 '''
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
124 Notes
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
125 =====
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
126
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
127 * I think we should have a FnPlugin decorator ( exactly like Olivier's) just
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
128 that also attaches the new created plugin to the schedule. This way you
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
129 can create plugin on the fly ( as long as they are simple functions that
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
130 print stuff, or compute simple statitics ).
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
131 * I added a method act to a Plugin. You use that to create the dependency
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
132 graph ( it could also be named listen to be more plugin like interface)
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
133 * Plugins are obtained in 3 ways :
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
134 - by wrapping a dataset / model or something similar
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
135 - by a function that constructs it from nothing
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
136 - by decorating a function
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
137 In all cases I would suggest then when creating them you should provide
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
138 the schedular as well, and the constructor also registers the plugin
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
139
1154
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
140 * The plugin concept works well as long as the plugins are a bit towards
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
141 heavy duty computation, disregarding printing plugins and such. If you have
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
142 many small plugins this system might only introduce an overhead. I would
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
143 argue that using theano is restricted to each plugin. Therefore I would
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
144 strongly suggest that the architecture to be done outside the schedular
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
145 with a different approach.
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
146
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
147 * I would suggest that the framework to be used only for the training loop
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
148 (after you get the adapt function, compute error function) so is more about
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
149 the meta-learner, hyper-learner learner level.
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
150
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
151 * A general remark that I guess everyone will agree on. We should make
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
152 sure that implementing a new plugin is as easy/simple as possible. We
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
153 have to hide all the complexity in the schedular ( it is the part of the
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
154 code we will not need or we would rarely need to work on).
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
155
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
156 * I have not went into how to implement the different components, but
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
157 following Olivier's code I think that part would be more or less straight
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
158 forward.
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
159
f923dddf0bf7 a better version of the script
pascanur
parents: 1153
diff changeset
160 '''
1153
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
161
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
162
ae5ba6206fd3 a first draft of pseudo-code for logreg .. using version B (?) approach
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
163 '''