Mercurial > pylearn
comparison doc/v2_planning/architecture.txt @ 1098:4eda3f52ebef
v2planning - revs to requirements, added architecture
| author | James Bergstra <bergstrj@iro.umontreal.ca> |
|---|---|
| date | Mon, 13 Sep 2010 09:42:36 -0400 |
| parents | |
| children | e5306f5626d4 |
comparison
equal
deleted
inserted
replaced
| 1097:8be7928cc1aa | 1098:4eda3f52ebef |
|---|---|
| 1 ==================== | |
| 2 Pylearn Architecture | |
| 3 ==================== | |
| 4 | |
| 5 | |
| 6 Basic Design Approach | |
| 7 ===================== | |
| 8 | |
| 9 I propose that the basic design of the library follow the Symbolic Expression | |
| 10 (SE) structure + virtual machine (VM) pattern that worked for Theano. | |
| 11 | |
| 12 So the main things for the library to provide would be: | |
| 13 | |
| 14 - a few VMs, some of which can run programs in parallel across processors, | |
| 15 hosts, and networks [R6,R8]; | |
| 16 | |
| 17 - MLA components as either individual Expressions (similar to Ops) or as | |
| 18 subgraphs of SEs [R5,R7,R10,R11] | |
| 19 | |
| 20 - machine learning algorithms including their training and testing in the form | |
| 21 of python functions that build SE graphs.[R1,R8]. | |
| 22 | |
| 23 This design addresses R2 (modularity) because swapping components is literally implemented by | |
| 24 swapping subgraphs. | |
| 25 | |
| 26 The design addresses R9 (algorithmic efficiency) because we can write | |
| 27 Theano-style graph transformations to recognize special cases of component | |
| 28 combinations. | |
| 29 | |
| 30 The design addresses R3 if we make the additional decision that the VMs (at | |
| 31 least sometimes) cache the return value of program function calls. This cache | |
| 32 serves as a database of experimental results, indexed by the functions that | |
| 33 originally computed them. I think this is a very natural scheme for organizing | |
| 34 experiment results, and ensuring experiment reproducibility [R1]. | |
| 35 At the same time, this is a clean and simple API behind which experiments can be | |
| 36 saved using a number of database technologies. | |
| 37 | |
| 38 APIs vs. lambda | |
| 39 ---------------- | |
| 40 | |
| 41 Modularity in general is achieved when pieces can be substituted one for the | |
| 42 other. | |
| 43 | |
| 44 In an object-oriented design, modularity is achieved by agreeing on interface | |
| 45 APIs, but in a functional design there is another possibility: the lambda. | |
| 46 | |
| 47 In an SE these pieces are expression [applications] and the subgraphs they form. | |
| 48 A subgraph is characterized syntactically within the program by its arguments | |
| 49 and its return values. A lambda function allows the User to create new | |
| 50 Expression types from arbitrary subgraphs with very few keystrokes. When a | |
| 51 lambda is available and easy to use, there is much less pressure on the | |
| 52 expression library to follow calling and return conventions strictly. | |
| 53 | |
| 54 Of course, the closer are two subgraphs in terms of their inputs, outputs, and | |
| 55 semantics, the easier it is to substitute one for the other. As library | |
| 56 designers, we should still aim for compatibility of similar algorithms. It's | |
| 57 just not essential to choose an API that will guarantee a match, or indeed to | |
| 58 choose any explicit API at all. | |
| 59 | |
| 60 |