Mercurial > pylearn

annotate doc/v2_planning/arch_src/plugin_JB_comments_RP.txt @ 1371:98d4232df1d8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
comment on OD idea
author Razvan Pascanu <r.pascanu@gmail.com>
date Mon, 15 Nov 2010 16:28:02 -0500
parents 699ed5f5f188
children
rev   line source
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
1 I agree with Ian, maybe using caps is not the best idea. It reminds be of BASIC which I used to do long time ago :). It also makes the code look a bit scary.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
2
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
3 JB replies: personally i think it makes the code look more AWESOME but I could
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
4 go either way. See reply to Ian in plugin_JB_comments_IG.txt
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
5
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
6 I like the approach and I think it goes close to my earliest proposition and to what I am proposing for the layer committeee ( though we did not have a meeting yet).
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
7 I would though write it in a more Theano like ( Ian has a example of how that would look). I would also drop the CALL and FILT constructs, and actually have a
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
8 decorator ( or something ) that wraps around a function to transform it into a call or filt. I hope that this is only syntactic sugar ( does this change anything
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
9 in the actual implementation ?? ) that makes things more natural. What I want to reach is something that looks very much as Theano, just that now you are creating
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
10 the graph of execution steps. Refractoring what you wrote this will look like
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
11
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
12 x = buffer_repeat( 1000, dataset.next())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
13 train_pca = pca.analyze(x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
14
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
15 train_pca.run()
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
16
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
17 If you allow a FILT to also get multiple inputs ( so not just the one) which comes natural in this way of writing you can get to describe a DAG that not only
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
18 describes the order of execution but also deals with what takes data from what. I'm sorry for not being there yesturday, from what I remember I have the
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
19 feeling that for you that is done under the hood and not taken care by this flow control structures.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
20
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
21 To be a bit more explicit, in the way of writing the code above you can see that :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
22 a) dataset_next() has to run before pca_analyze
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
23 b) pca_analyze needs the result (data) object of buffer_repeat( dataset.next())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
24
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
25 I've actually elaborated on this idea here and there, and figured out what the result from such a control flow thing is, and how to make everything explicit
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
26 in the graph. Parts of this is in my plugin_RP.py ( Step 1) though it is a bit of a moving target. I also have a sligtly different way of writing REPEAT
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
27 and BUFFER_REPEAT .. though I think is mostly the same. I actually did not know how to deal with distributed things until I saw how you deal with that in your code.
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
28 Copy-pasted a version of a SDAA with my way of writing :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
29
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
30 ## Layer 1:
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
31
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
32 data_x,data_y = GPU_transform(load_mnist())
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
33 noisy_data_x = gaussian_noise(data_x, amount = 0.1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
34 hidden1 = tanh(dotW_b(data_x, n_units = 200))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
35 reconstruct1 = reconstruct(hidden1.replace(data_x, noisy_data_x),
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
36 noisy_data_x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
37 err1 = cross_entropy(reconstruct1, data_x)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
38 learner1 = SGD(err1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
39
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
40 # Layer 2 :
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
41 noisy_hidden1 = gaussian_noise(hidden1, amount = 0.1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
42 hidden2 = tanh(dotW_b(hidden1, n_units = 200))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
43 reconstruct2 = reconstruct(hidden2.replace(hidden1,noisy_hidden1),
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
44 noisy_hidden1)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
45 err2 = cross_entropy(reconstruct2, hidden)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
46 learner2 = SGD(err2)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
47
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
48 # Top layer:
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
49
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
50 output = sigmoid(dotW_b(hidden2, n_units = 10))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
51 err = cross_entropy(output, data_y)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
52 learner = SGD(err)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
53
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
54
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
55 GPU_transform,gaussian_noise and so on are functions that have been decorated ( or classes if you want)
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
56 that you would write using FILT. Reconstruct for me is a different CONTROL FLOW element.
1214
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
57 In this case I don't use REPEAT or BUFFER_REPEAT or the other very cool control flow elements, but you
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
58 can easily imagine writing something like
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
59
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
60 pretrained_in_parallel = weave( learner1, learner2)
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
61 results = spawn(repeat(5000,learner1),repeat(500,learner2))
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
62
681b5e7e3b81 a few comments on James version
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
63
1216
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
64 JB replies:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
65
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
66 This reply makes it clearer to me that I was not sensitive enough to the
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
67 difference between *expressions* and *control-flow statements*. What you have
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
68 above is a graph of declarative expressions (if I understand correctly) with
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
69 certain properties:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
70
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
71 - they have no side effects
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
72 - they can be re-ordered within dependency constraints
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
73
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
74 Contrast this with the CALL statements in my proposal:
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
75
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
76 - they work primarily by side effect
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
77 - they cannot be re-ordered at all
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
78
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
79 So the fact that CALL currently works by side effect means that there is
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
80 almost no graph-manipulation that can be guaranteed not to change the program.
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
81 This is a reason to make CALL statements *encapsulate* programs constructed
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
82 using declarative constructs (i.e. Theano functions)
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
83
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
84 In other words, in this short term, this feels to me like the reason to *not*
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
85 mix Theano graph building with this control-flow business.
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
86
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
87 Consequently, I think I will remove the BUFFER_REPEAT construct since that is
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
88 really an expression masquerading as a control flow statement, and I will
5a8930e089ed replied in plugin_JB_comments_RP
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1214
diff changeset
89 remove FILT too.
1218
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
90
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
91 RP asks:
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
92
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
93 I understand now the difference between what you wrote and what I had in
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
94 mind. Though I don't undestand the argument against it. Do you mean to say
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
95 that writing it the way I proposed implies a much more complicated backbone
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
96 framework which will take us to long to develop? Or is there something else
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
97 that you meant ?
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
98
5d1b5906151c I did not fully get the argument against what I wrote ..
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1216
diff changeset
99
1220
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
100 JB replies:
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
101
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
102 I don't think it's necessary to combine theano with this control-flow
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
103 proposal, and I don't know how to do it. Yes, it seems like it would be hard
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
104 and/or awkward, and I don't even really see the advantage of even trying to do
35fb6e9713d2 reply to razvan re: plugin_jb
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1218
diff changeset
105 it.
1221
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
106
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
107 RP:
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
108
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
109 I think you misunderstood me. I did not propose to mix Theano with the
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
110 library. I agree that would be awkward. What I had in mind ( which might be
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
111 just something different from what you are doing) is to use some concepts
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
112 from how Theano deals with things.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
113
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
114 For example right now you added registers. Writing something like:
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
115
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
116 CALL( fn, arg1, arg2, arg3, _set= reg('x') )
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
117
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
118 means actually reg('x') = fn (arg1,arg2,arg3)
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
119
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
120 You get most of what you want because this control flow elements don't
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
121 actually get executed until you run the program. That means that you
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
122 have a fixed simple graph ( you can't play around with it) that tells
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
123 how to execute your commands. You can save that graph, and the point in
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
124 the graph where you stop so that you can resume latter. You can also
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
125 save all registers at that point.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
126
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
127
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
128 Why not have that fn instead of being a python function, be some class
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
129 that implements a method run which does what your call would do.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
130 The init/ or __call__ of that class would do what CALL does in your case.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
131 Do you think that would be impossible to implement? Any such function
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
132 could either return a set of registers or not.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
133
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
134 Your other control flow things will be just special such functions.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
135 The only thing that would might look a bit strange would be the sequence
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
136 in case you need to return things. Maybe I could use the same trick,
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
137 namely a _set arguemnt to __call__.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
138
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
139 I'm not against your approach, I just think it can be written a bit
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
140 differently, which in my opinion is easier to read, understand and so
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
141 on. I will have nothing against if we decide to write it exactly how you
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
142 propose and I'm sure that I will get the hang of it pretty fast.
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
143
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
144
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
145 Bottom line (in my view):
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
146 - I don't say we should mix Theano with anything
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
147 - I think writing things such that it looks like applying functions to
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
148 object is a more natural way, easy to understand for noobs
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
149 - Writing a new functions by inheriting a class and implementing a method
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
150 is also natural
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
151 - I do not propose to do optimizations or play with the graph ! I do
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
152 though think that you should be able to :
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
153 * replace parts of a subgraph with a different
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
154 * automatically collect hyper-parameters or parameters if you ever want
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
155 to
699ed5f5f188 answer to James comment
Razvan Pascanu <r.pascanu@gmail.com>
parents: 1220
diff changeset
156 * change the value of these somehow