Mercurial > pylearn

annotate doc/v2_planning/arch_src/plugin_JB_comments_RP.txt @ 1224:f68b857eb11b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
reply to comments on architecture document
author Ian Goodfellow
date Wed, 22 Sep 2010 15:44:14 -0400
parents 699ed5f5f188
children
rev   line source
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
1 I agree with Ian, maybe using caps is not the best idea. It reminds be of BASIC which I used to do long time ago :). It also makes the code look a bit scary.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
2
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
3 JB replies: personally i think it makes the code look more AWESOME but I could
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
4 go either way. See reply to Ian in plugin_JB_comments_IG.txt
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
5
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
6 I like the approach and I think it goes close to my earliest proposition and to what I am proposing for the layer committeee ( though we did not have a meeting yet).
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
7 I would though write it in a more Theano like ( Ian has a example of how that would look). I would also drop the CALL and FILT constructs, and actually have a
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
8 decorator ( or something ) that wraps around a function to transform it into a call or filt. I hope that this is only syntactic sugar ( does this change anything
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
9 in the actual implementation ?? ) that makes things more natural. What I want to reach is something that looks very much as Theano, just that now you are creating
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
10 the graph of execution steps. Refractoring what you wrote this will look like
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
11
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
12 x = buffer_repeat( 1000, dataset.next())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
13 train_pca = pca.analyze(x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
14
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
15 train_pca.run()
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
16
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
17 If you allow a FILT to also get multiple inputs ( so not just the one) which comes natural in this way of writing you can get to describe a DAG that not only
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
18 describes the order of execution but also deals with what takes data from what. I'm sorry for not being there yesturday, from what I remember I have the
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
19 feeling that for you that is done under the hood and not taken care by this flow control structures.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
20
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
21 To be a bit more explicit, in the way of writing the code above you can see that :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
22 a) dataset_next() has to run before pca_analyze
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
23 b) pca_analyze needs the result (data) object of buffer_repeat( dataset.next())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
24
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
25 I've actually elaborated on this idea here and there, and figured out what the result from such a control flow thing is, and how to make everything explicit
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
26 in the graph. Parts of this is in my plugin_RP.py ( Step 1) though it is a bit of a moving target. I also have a sligtly different way of writing REPEAT
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
27 and BUFFER_REPEAT .. though I think is mostly the same. I actually did not know how to deal with distributed things until I saw how you deal with that in your code.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
28 Copy-pasted a version of a SDAA with my way of writing :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
29
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
30 ## Layer 1:
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
31
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
32 data_x,data_y = GPU_transform(load_mnist())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
33 noisy_data_x = gaussian_noise(data_x, amount = 0.1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
34 hidden1 = tanh(dotW_b(data_x, n_units = 200))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
35 reconstruct1 = reconstruct(hidden1.replace(data_x, noisy_data_x),
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
36 noisy_data_x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
37 err1 = cross_entropy(reconstruct1, data_x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
38 learner1 = SGD(err1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
39
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
40 # Layer 2 :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
41 noisy_hidden1 = gaussian_noise(hidden1, amount = 0.1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
42 hidden2 = tanh(dotW_b(hidden1, n_units = 200))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
43 reconstruct2 = reconstruct(hidden2.replace(hidden1,noisy_hidden1),
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
44 noisy_hidden1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
45 err2 = cross_entropy(reconstruct2, hidden)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
46 learner2 = SGD(err2)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
47
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
48 # Top layer:
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
49
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
50 output = sigmoid(dotW_b(hidden2, n_units = 10))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
51 err = cross_entropy(output, data_y)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
52 learner = SGD(err)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
53
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
54
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
55 GPU_transform,gaussian_noise and so on are functions that have been decorated ( or classes if you want)
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
56 that you would write using FILT. Reconstruct for me is a different CONTROL FLOW element.
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
57 In this case I don't use REPEAT or BUFFER_REPEAT or the other very cool control flow elements, but you
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
58 can easily imagine writing something like
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
59
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
60 pretrained_in_parallel = weave( learner1, learner2)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
61 results = spawn(repeat(5000,learner1),repeat(500,learner2))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
62
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
63
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
64 JB replies:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
65
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
66 This reply makes it clearer to me that I was not sensitive enough to the
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
67 difference between *expressions* and *control-flow statements*. What you have
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
68 above is a graph of declarative expressions (if I understand correctly) with
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
69 certain properties:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
70
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
71 - they have no side effects
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
72 - they can be re-ordered within dependency constraints
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
73
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
74 Contrast this with the CALL statements in my proposal:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
75
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
76 - they work primarily by side effect
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
77 - they cannot be re-ordered at all
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
78
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
79 So the fact that CALL currently works by side effect means that there is
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
80 almost no graph-manipulation that can be guaranteed not to change the program.
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
81 This is a reason to make CALL statements *encapsulate* programs constructed
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
82 using declarative constructs (i.e. Theano functions)
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
83
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
84 In other words, in this short term, this feels to me like the reason to *not*
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
85 mix Theano graph building with this control-flow business.
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
86
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
87 Consequently, I think I will remove the BUFFER_REPEAT construct since that is
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
88 really an expression masquerading as a control flow statement, and I will
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
89 remove FILT too.
1218
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
90
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
91 RP asks:
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
92
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
93 I understand now the difference between what you wrote and what I had in
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
94 mind. Though I don't undestand the argument against it. Do you mean to say
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
95 that writing it the way I proposed implies a much more complicated backbone
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
96 framework which will take us to long to develop? Or is there something else
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
97 that you meant ?
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
98
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
99
1220
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
100 JB replies:
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
101
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
102 I don't think it's necessary to combine theano with this control-flow
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
103 proposal, and I don't know how to do it. Yes, it seems like it would be hard
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
104 and/or awkward, and I don't even really see the advantage of even trying to do
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
105 it.
1221
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
106
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
107 RP:
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
108
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
109 I think you misunderstood me. I did not propose to mix Theano with the
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
110 library. I agree that would be awkward. What I had in mind ( which might be
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
111 just something different from what you are doing) is to use some concepts
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
112 from how Theano deals with things.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
113
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
114 For example right now you added registers. Writing something like:
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
115
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
116 CALL( fn, arg1, arg2, arg3, _set= reg('x') )
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
117
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
118 means actually reg('x') = fn (arg1,arg2,arg3)
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
119
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
120 You get most of what you want because this control flow elements don't
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
121 actually get executed until you run the program. That means that you
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
122 have a fixed simple graph ( you can't play around with it) that tells
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
123 how to execute your commands. You can save that graph, and the point in
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
124 the graph where you stop so that you can resume latter. You can also
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
125 save all registers at that point.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
126
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
127
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
128 Why not have that fn instead of being a python function, be some class
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
129 that implements a method run which does what your call would do.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
130 The init/ or __call__ of that class would do what CALL does in your case.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
131 Do you think that would be impossible to implement? Any such function
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
132 could either return a set of registers or not.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
133
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
134 Your other control flow things will be just special such functions.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
135 The only thing that would might look a bit strange would be the sequence
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
136 in case you need to return things. Maybe I could use the same trick,
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
137 namely a _set arguemnt to __call__.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
138
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
139 I'm not against your approach, I just think it can be written a bit
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
140 differently, which in my opinion is easier to read, understand and so
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
141 on. I will have nothing against if we decide to write it exactly how you
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
142 propose and I'm sure that I will get the hang of it pretty fast.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
143
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
144
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
145 Bottom line (in my view):
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
146 - I don't say we should mix Theano with anything
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
147 - I think writing things such that it looks like applying functions to
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
148 object is a more natural way, easy to understand for noobs
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
149 - Writing a new functions by inheriting a class and implementing a method
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
150 is also natural
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
151 - I do not propose to do optimizations or play with the graph ! I do
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
152 though think that you should be able to :
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
153 * replace parts of a subgraph with a different
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
154 * automatically collect hyper-parameters or parameters if you ever want
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
155 to
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
156 * change the value of these somehow