Mercurial > pylearn

comparison doc/v2_planning/arch_src/plugin_JB_comments_RP.txt @ 1216:5a8930e089ed

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
replied in plugin_JB_comments_RP
author James Bergstra <bergstrj@iro.umontreal.ca>
date Wed, 22 Sep 2010 12:12:30 -0400
parents 681b5e7e3b81
children 5d1b5906151c
comparison
equal deleted inserted replaced
1215:4754661ad6ab 1216:5a8930e089ed
1 I agree with Ian, maybe using caps is not the best idea. It reminds be of BASIC which I used to do long time ago :). It also makes the code look a bit scary. 1 I agree with Ian, maybe using caps is not the best idea. It reminds be of BASIC which I used to do long time ago :). It also makes the code look a bit scary.
2 2
3 JB replies: personally i think it makes the code look more AWESOME but I could
4 go either way. See reply to Ian in plugin_JB_comments_IG.txt
5
3 I like the approach and I think it goes close to my earliest proposition and to what I am proposing for the layer committeee ( though we did not have a meeting yet). 6 I like the approach and I think it goes close to my earliest proposition and to what I am proposing for the layer committeee ( though we did not have a meeting yet).
4 I would though write it in a more Theano like ( Ian has a example of how that would look). I would also drop the CALL and FLIT constructs, and actually have a 7 I would though write it in a more Theano like ( Ian has a example of how that would look). I would also drop the CALL and FILT constructs, and actually have a
5 decorator ( or something ) that wraps around a function to transform it into a call or flit. I hope that this is only syntactic sugar ( does this change anything 8 decorator ( or something ) that wraps around a function to transform it into a call or filt. I hope that this is only syntactic sugar ( does this change anything
6 in the actual implementation ?? ) that makes things more natural. What I want to reach is something that looks very much as Theano, just that now you are creating 9 in the actual implementation ?? ) that makes things more natural. What I want to reach is something that looks very much as Theano, just that now you are creating
7 the graph of execution steps. Refractoring what you wrote this will look like 10 the graph of execution steps. Refractoring what you wrote this will look like
8 11
9 x = buffer_repeat( 1000, dataset.next()) 12 x = buffer_repeat( 1000, dataset.next())
10 train_pca = pca.analyze(x) 13 train_pca = pca.analyze(x)
11 14
12 train_pca.run() 15 train_pca.run()
13 16
14 If you allow a FLIT to also get multiple inputs ( so not just the one) which comes natural in this way of writing you can get to describe a DAG that not only 17 If you allow a FILT to also get multiple inputs ( so not just the one) which comes natural in this way of writing you can get to describe a DAG that not only
15 describes the order of execution but also deals with what takes data from what. I'm sorry for not being there yesturday, from what I remember I have the 18 describes the order of execution but also deals with what takes data from what. I'm sorry for not being there yesturday, from what I remember I have the
16 feeling that for you that is done under the hood and not taken care by this flow control structures. 19 feeling that for you that is done under the hood and not taken care by this flow control structures.
17 20
18 To be a bit more explicit, in the way of writing the code above you can see that : 21 To be a bit more explicit, in the way of writing the code above you can see that :
19 a) dataset_next() has to run before pca_analyze 22 a) dataset_next() has to run before pca_analyze
48 err = cross_entropy(output, data_y) 51 err = cross_entropy(output, data_y)
49 learner = SGD(err) 52 learner = SGD(err)
50 53
51 54
52 GPU_transform,gaussian_noise and so on are functions that have been decorated ( or classes if you want) 55 GPU_transform,gaussian_noise and so on are functions that have been decorated ( or classes if you want)
53 that you would write using FLIT. Reconstruct for me is a different CONTROL FLOW element. 56 that you would write using FILT. Reconstruct for me is a different CONTROL FLOW element.
54 In this case I don't use REPEAT or BUFFER_REPEAT or the other very cool control flow elements, but you 57 In this case I don't use REPEAT or BUFFER_REPEAT or the other very cool control flow elements, but you
55 can easily imagine writing something like 58 can easily imagine writing something like
56 59
57 pretrained_in_parallel = weave( learner1, learner2) 60 pretrained_in_parallel = weave( learner1, learner2)
58 results = spawn(repeat(5000,learner1),repeat(500,learner2)) 61 results = spawn(repeat(5000,learner1),repeat(500,learner2))
59 62
60 63
64 JB replies:
65
66 This reply makes it clearer to me that I was not sensitive enough to the
67 difference between *expressions* and *control-flow statements*. What you have
68 above is a graph of declarative expressions (if I understand correctly) with
69 certain properties:
70
71 - they have no side effects
72 - they can be re-ordered within dependency constraints
73
74 Contrast this with the CALL statements in my proposal:
75
76 - they work primarily by side effect
77 - they cannot be re-ordered at all
78
79 So the fact that CALL currently works by side effect means that there is
80 almost no graph-manipulation that can be guaranteed not to change the program.
81 This is a reason to make CALL statements *encapsulate* programs constructed
82 using declarative constructs (i.e. Theano functions)
83
84 In other words, in this short term, this feels to me like the reason to *not*
85 mix Theano graph building with this control-flow business.
86
87 Consequently, I think I will remove the BUFFER_REPEAT construct since that is
88 really an expression masquerading as a control flow statement, and I will
89 remove FILT too.
90