Mercurial > pylearn

comparison doc/v2_planning/architecture.txt @ 1115:967975f9c574

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added Jobman compatibility to architecture.txt
author James Bergstra <bergstrj@iro.umontreal.ca>
date Tue, 14 Sep 2010 13:41:48 -0400
parents e5306f5626d4
children 5387666d49b4
comparison
equal deleted inserted replaced
1114:de153244c8e5 1115:967975f9c574
1 ==================== 1 ====================
2 Pylearn Architecture 2 Pylearn Architecture
3 ==================== 3 ====================
4 4
5 5
6 Basic Design Approach 6 SE + VM Approach
7 ===================== 7 =================
8 8
9 I propose that the basic design of the library follow the Symbolic Expression 9 One avenue for the basic design of the library is to follow the Symbolic
10 (SE) structure + virtual machine (VM) pattern that worked for Theano. 10 Expression (SE) structure + virtual machine (VM) pattern that worked for Theano.
11 11
12 So the main things for the library to provide would be: 12 The main things for the library to provide would be:
13 13
14 - a few VMs, some of which can run programs in parallel across processors, 14 - a few VMs, some of which can run programs in parallel across processors,
15 hosts, and networks [R6,R8]; 15 hosts, and networks [R6,R8];
16 16
17 - MLA components as either individual Expressions (similar to Ops) or as 17 - MLA components as either individual Expressions (similar to Ops) or as
107 (*) Note that I cheated a bit in my example above: the graph from the 107 (*) Note that I cheated a bit in my example above: the graph from the
108 encapsulated version is not a simple chain, so it is not obvious how to 108 encapsulated version is not a simple chain, so it is not obvious how to
109 convert it into the pipeline given in the linear version. It's still possible 109 convert it into the pipeline given in the linear version. It's still possible
110 though, but this is probably not the place to get into the details. 110 though, but this is probably not the place to get into the details.
111 111
112
113
114 Jobman Compatibility Approach
115 =============================
116
117 One basic approach for the library is to provide a set of components that are
118 compatible with remote execution. The emphasis could be not so much on
119 standardizing the roles and APIs of components, so much as ensuring that they
120 can be glued together and supports parallel execution on one or more CPUs or
121 clusters.
122
123 In this approach we would provide a proxy for asynchronous execution
124 (e.g. "pylearn.call(fn, args, kwargs, backend=default_backend)"), which would
125 come with constraints on what fn, args, and kwargs can be. Specifically, they
126 must be picklable, and there are benefits (e.g. automatic function call caching)
127 associated with their being hashable as well.
128
129
130