Mercurial > pylearn
changeset 1115:967975f9c574
added Jobman compatibility to architecture.txt
author | James Bergstra <bergstrj@iro.umontreal.ca> |
---|---|
date | Tue, 14 Sep 2010 13:41:48 -0400 |
parents | de153244c8e5 |
children | 18a092001752 |
files | doc/v2_planning/architecture.txt |
diffstat | 1 files changed, 24 insertions(+), 5 deletions(-) [+] |
line wrap: on
line diff
--- a/doc/v2_planning/architecture.txt Tue Sep 14 13:33:45 2010 -0400 +++ b/doc/v2_planning/architecture.txt Tue Sep 14 13:41:48 2010 -0400 @@ -3,13 +3,13 @@ ==================== -Basic Design Approach -===================== +SE + VM Approach +================= -I propose that the basic design of the library follow the Symbolic Expression -(SE) structure + virtual machine (VM) pattern that worked for Theano. +One avenue for the basic design of the library is to follow the Symbolic +Expression (SE) structure + virtual machine (VM) pattern that worked for Theano. -So the main things for the library to provide would be: +The main things for the library to provide would be: - a few VMs, some of which can run programs in parallel across processors, hosts, and networks [R6,R8]; @@ -109,3 +109,22 @@ convert it into the pipeline given in the linear version. It's still possible though, but this is probably not the place to get into the details. + + +Jobman Compatibility Approach +============================= + +One basic approach for the library is to provide a set of components that are +compatible with remote execution. The emphasis could be not so much on +standardizing the roles and APIs of components, so much as ensuring that they +can be glued together and supports parallel execution on one or more CPUs or +clusters. + +In this approach we would provide a proxy for asynchronous execution +(e.g. "pylearn.call(fn, args, kwargs, backend=default_backend)"), which would +come with constraints on what fn, args, and kwargs can be. Specifically, they +must be picklable, and there are benefits (e.g. automatic function call caching) +associated with their being hashable as well. + + +