Mercurial > pylearn
diff doc/v2_planning/learner.txt @ 1044:3b1fd599bafd
my first draft of my own views which are close to be just a reformulation of what James proposes
author | Razvan Pascanu <r.pascanu@gmail.com> |
---|---|
date | Wed, 08 Sep 2010 12:55:30 -0400 |
parents | 3f528656855b |
children | d57bdd9a9980 |
line wrap: on
line diff
--- a/doc/v2_planning/learner.txt Wed Sep 08 11:33:33 2010 -0400 +++ b/doc/v2_planning/learner.txt Wed Sep 08 12:55:30 2010 -0400 @@ -267,3 +267,111 @@ during graph exploration and (b) learner instances are big, and we don't want to have to keep a whole saved model just to attach meta-info e.g. validation score. Choosing this API spills over into other committees, so we should get their feedback about how to resolve it. + +Just another view/spin on the same idea (Razvan) +================================================ + + +My idea is probably just a spin off from what James wrote. It is an extension +of what I send on the mailing list some time ago. + +Big Picture +----------- + +What do we care about ? +~~~~~~~~~~~~~~~~~~~~~~~ + +This is the list of the main points that I have in mind : + + * Re-usability + * Extensibility + * Simplicity or easily readable code ( connected to re-usability ) + * Modularity ( connected to extensibility ) + * Fast to write code ( - sort of comes out of simplicity) + * Efficient code + + +Composition +~~~~~~~~~~~ + +To me this reads as code generated by composing pieces. Imagine this : +you start of with something primitive that I will call a "variable", which +probably is a very unsuitable name. And then you compose those intial +"variables" or transform them through several "functions". Each such +"function" hides some logic, that you as the user don't care about. +You can have low-level or micro "functions" and high-level or macro +"functions", where a high-level function is just a certain compositional +pattern of low-level "functions". There are several classes of "functions" +and "variables" that can be inter-changable. This is how modularity is +obtained, by chainging between functions from a certain class. + +Now when you want to research something, what you do is first select +the step you want to look into. If you are lucky you can re-write this +step as certain decomposition of low-level transformations ( there can be +multiple such decompositions). If not you have to implement such a +decompositions acording to your needs. Pick the low-level transformations you want +to change and write new versions that implement your logic. + +I think the code will be easy to read, because it is just applying a fixed +set of transformations, one after the other. The one who writes the code can +decide how explicit he wants to write things by switching between high-level +and low-level functions. + +I think the code this way is re-usable, because you can just take this chain +of transformation and replace the one you care about, without looking into +the rest. + +You get this fractal property of the code. Zooming in, you always get just +a set of functions applied to a set of variables. In the begining those might +not be there, and you would have to create new "low level" decompositions, +maybe even new "variables" that get data between those decompositions. + +The thing with variables here, is that I don't want this "functions" to have +a state. All the information is passed along through these variables. This +way understanding the graph is easy, debugging it is also easier ( then having +all these hidden states ..) + +Note that while doing so we might ( and I strongly think we should) create +a (symbolic) DAG of operations. ( this is where it becomes what James was saying). +In such a DAG the "variables" will the nodes and the functions will be edges. +I think having a DAG is useful in many ways (all this are things that one +might think about implementing in a far future, I'm not proposing to implement +them unless we want to use them - like the reconstruction ): + * there exist the posibility of writing optimizations ( theano style ) + * there exist the posibility to add global view utility functions ( like + a reconstruction function for SdA - extremely low level here), or global + view diagnostic tools + * the posibility of creating a GUI ( where you just create the Graph by + picking transforms and variables from a list ) or working interactively + and then generating code that will reproduce the graph + * you can view the graph and different granularity levels to understand + things ( global diagnostics) + +We should have a taxonomy of possible classes of functions and possible +classes of variables, but those should not be exclusive. We can work at a high +level for now, and decompose those high level functions to lower level when +we need to. We can introduce new classes of functions or intermediate +variables between those low level functions. + + +Similarities with James' idea +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + As I said before, this is I think just another view on what James proposed. + The learner in his case is the module that traverses the graph of this + operations, which makes sense here as well. + + The 'execute' command in his api is just applying a function to some variables in + my case. + + The learner keeps track of the graph that is formed I think in both cases. + + His view is a bit more general. I see the graph as fully created by the user, + and the learner just has to go from the start to the end. In his case the + traversal is conditioned on some policies. I think these ideas can be mixed / + united. What I would see in my case to have this functionality is something + similar to the lazy linker for Theano. + + + +