diff doc/v2_planning/learner.txt @ 1044:3b1fd599bafd

my first draft of my own views which are close to be just a reformulation of what James proposes
author Razvan Pascanu <r.pascanu@gmail.com>
date Wed, 08 Sep 2010 12:55:30 -0400
parents 3f528656855b
children d57bdd9a9980
line wrap: on
line diff
--- a/doc/v2_planning/learner.txt	Wed Sep 08 11:33:33 2010 -0400
+++ b/doc/v2_planning/learner.txt	Wed Sep 08 12:55:30 2010 -0400
@@ -267,3 +267,111 @@
 during graph exploration and (b) learner instances are big, and we don't want to have to keep a
 whole saved model just to attach meta-info e.g. validation score.    Choosing this API spills
 over into other committees, so we should get their feedback about how to resolve it.
+
+Just another view/spin on the same idea (Razvan)
+================================================
+
+
+My idea is probably just a spin off from what James wrote. It is an extension
+of what I send on the mailing list some time ago.
+
+Big Picture
+-----------
+
+What do we care about ?
+~~~~~~~~~~~~~~~~~~~~~~~
+
+This is the list of the main points that I have in mind :
+
+ * Re-usability 
+ * Extensibility
+ * Simplicity or easily readable code ( connected to re-usability )
+ * Modularity ( connected to extensibility )
+ * Fast to write code ( - sort of comes out of simplicity)
+ * Efficient code
+
+
+Composition
+~~~~~~~~~~~
+
+To me this reads as code generated by composing pieces. Imagine this :
+you start of with something primitive that I will call a "variable", which
+probably is a very unsuitable name. And then you compose those intial
+"variables" or transform them through several "functions". Each such 
+"function" hides some logic, that you as the user don't care about. 
+You can have low-level or micro "functions" and high-level or macro 
+"functions", where a high-level function is just a certain compositional
+pattern of low-level "functions". There are several classes of "functions"
+and "variables" that can be inter-changable. This is how modularity is 
+obtained, by chainging between functions from a certain class.
+
+Now when you want to research something, what you do is first select
+the step you want to look into. If you are lucky you can re-write this
+step as certain decomposition of low-level transformations ( there can be
+multiple such decompositions). If not you have to implement such a 
+decompositions acording to your needs. Pick the low-level transformations you want 
+to change and write new versions that implement your logic. 
+
+I think the code will be easy to read, because it is just applying a fixed
+set of transformations, one after the other. The one who writes the code can
+decide how explicit he wants to write things by switching between high-level
+and low-level functions.
+
+I think the code this way is re-usable, because you can just take this chain
+of transformation and replace the one you care about, without looking into 
+the rest. 
+
+You get this fractal property of the code. Zooming in, you always get just 
+a set of functions applied to a set of variables. In the begining those might 
+not be there, and you would have to create new "low level" decompositions,
+maybe even new "variables" that get data between those decompositions.
+
+The thing with variables here, is that I don't want this "functions" to have
+a state. All the information is passed along through these variables. This 
+way understanding the graph is easy, debugging it is also easier ( then having
+all these hidden states ..)
+
+Note that while doing so we might ( and I strongly think we should) create 
+a (symbolic) DAG of operations. ( this is where it becomes what James was saying).
+In such a DAG the "variables" will the nodes and the functions will be edges.
+I think having a DAG is useful in many ways (all this are things that one
+might think about implementing in a far future, I'm not proposing to implement
+them unless we want to use them - like the reconstruction ):
+  * there exist the posibility of writing optimizations ( theano style ) 
+  * there exist the posibility to add global view utility functions ( like 
+  a reconstruction function for SdA - extremely low level here), or global 
+  view diagnostic tools
+  * the posibility of creating a GUI ( where you just create the Graph by
+  picking transforms and variables from a list ) or working interactively
+  and then generating code that will reproduce the graph 
+  * you can view the graph and different granularity levels to understand
+  things ( global diagnostics)
+
+We should have a taxonomy of possible classes of functions and possible
+classes of variables, but those should not be exclusive. We can work at a high
+level for now, and decompose those high level functions to lower level when 
+we need to. We can introduce new classes of functions or intermediate
+variables between those low level functions.
+
+
+Similarities with James' idea
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ As I said before, this is I think just another view on what James proposed.
+ The learner in his case is the module that traverses the graph of this
+ operations, which makes sense here as well. 
+ 
+ The  'execute' command in his api is just applying a function to some variables in 
+ my case. 
+
+ The learner keeps track of the graph that is formed I think in both cases.
+ 
+ His view is a bit more  general. I see the graph as fully created by the user, 
+ and the learner just has to go from the start to the end. In his case the
+ traversal is conditioned on some policies. I think these ideas can be mixed / 
+ united. What I would see in my case to have this functionality is something
+ similar to the lazy linker for Theano.
+ 
+
+
+