Mercurial > pylearn
comparison doc/v2_planning/arch_src/plugin_JB_comments_YB.txt @ 1247:8dfe9d6e72f6
plugin_JB replies
| author | James Bergstra <bergstrj@iro.umontreal.ca> |
|---|---|
| date | Thu, 23 Sep 2010 13:20:19 -0400 |
| parents | 808e38dce8d6 |
| children | ab1db1837e98 |
comparison
equal
deleted
inserted
replaced
| 1246:14444845989a | 1247:8dfe9d6e72f6 |
|---|---|
| 11 Disadvantages: | 11 Disadvantages: |
| 12 | 12 |
| 13 * much more difficult to read | 13 * much more difficult to read |
| 14 * much more difficult to debug | 14 * much more difficult to debug |
| 15 | 15 |
| 16 JB asks: I would like to try and correct you, but I don't know where to begin -- | |
| 17 - What do you think is more difficult to read [than what?] and why? | |
| 18 - What do you expect to be more difficult [than what?] to debug? | |
| 19 | |
| 20 | |
| 16 Advantages: | 21 Advantages: |
| 17 | 22 |
| 18 * easier to serialize (can't we serialize an ordinary Python class created by a normal user?) | 23 * easier to serialize (can't we serialize an ordinary Python class created by a normal user?) |
| 19 * possible but not easier to programmatically modify existing learning algorithms | 24 * possible but not easier to programmatically modify existing learning algorithms |
| 20 (why not the usual constructor parameters and hooks, | 25 (why not the usual constructor parameters and hooks, |
| 21 when possible, and just create another code for a new DBN variant when it can't fit?) | 26 when possible, and just create another code for a new DBN variant when it can't fit?) |
| 22 * am I missing something? | 27 * am I missing something? |
| 28 | |
| 29 JB replies: | |
| 30 - Re serializibility - I think any system that supports program pausing, | |
| 31 resuming, and dynamic editing (a.k.a. process surgery) will have the flavour | |
| 32 of my proposal. If someone has a better idea, he should suggest it. | |
| 33 | |
| 34 - Re hooks & constructors - the mechanism I propose is more flexible than hooks and constructor | |
| 35 parameters. Hooks and constructor parameters have their place, and would be | |
| 36 used under my proposal as usual to configure the modules on which the | |
| 37 flow-graph operates. But flow-graphs are more flexible. Flow-graphs | |
| 38 (REPEAT, CALL, etc.) that are constructed by library calls can be directly | |
| 39 modified. You can add new hooks, for example, or add a print statement | |
| 40 between two statements (CALLs) that previously had no hook between them. | |
| 41 - the analagous thing using the real python VM would be to dynamically | |
| 42 re-program Python's compiled bytecode, which I don't think is possible. | |
| 23 | 43 |
| 24 I am not convinced that any of the stated advantages can't be achieved in more traditional ways. | 44 I am not convinced that any of the stated advantages can't be achieved in more traditional ways. |
| 25 | 45 |
| 26 RP comment: James or anybody else correct me if I'm wrong. What I think James | 46 RP comment: James or anybody else correct me if I'm wrong. What I think James |
| 27 proposed is just a way encapsulating different steps of the program in some | 47 proposed is just a way encapsulating different steps of the program in some |
| 53 want to use everywhere. As far as serialization is concerned, I think this | 73 want to use everywhere. As far as serialization is concerned, I think this |
| 54 should be do-able without such a system (provided we all agree that we do not | 74 should be do-able without such a system (provided we all agree that we do not |
| 55 necessarily require the ability to serialize / restart at any point). About | 75 necessarily require the ability to serialize / restart at any point). About |
| 56 the ability to move / substitute things, you could probably achieve the same | 76 the ability to move / substitute things, you could probably achieve the same |
| 57 goal with proper code factorization / conventions. | 77 goal with proper code factorization / conventions. |
| 78 | |
| 79 JB replies: | |
| 80 You are right that with sufficient discipline on everyone's part, | |
| 81 and a good design using existing python control flow (loops and functions) it is | |
| 82 probably possible to get many of the features I'm claiming with my proposal. | |
| 83 | |
| 84 But I don't think Python offers a very helpful syntax or control flow elements | |
| 85 for programming parallel distributed computations through, because the python | |
| 86 interpreter doesn't do that. | |
| 87 | |
| 88 What I'm trying to design is a mechanism that can allow us to *express the entire | |
| 89 learning algorithm* in a program. That means | |
| 90 - including the grid-search, | |
| 91 - including the use of the cluster, | |
| 92 - including the pre-processing and post-processing. | |
| 93 | |
| 94 To make that actually work, programs need to be more flexible - we need to be | |
| 95 able to pause and resume 'function calls', and to possibly restart them if we | |
| 96 find a problem (without having to restart the whole program). We already do | |
| 97 these things in ad-hoc ways by writing scripts, generating intermediate files, | |
| 98 etc., but I think we would empower ourselves by using a tool that lets us | |
| 99 actually write down the *WHOLE* algorithm, in one place rather than as a README | |
| 100 with a list of scripts and instructions for what to do with them (especially | |
| 101 because the README often never gets written). | |
| 102 |