--- a/doc/v2_planning/learner.txt	Thu Sep 09 11:28:24 2010 -0400
+++ b/doc/v2_planning/learner.txt	Thu Sep 09 11:50:37 2010 -0400
@@ -256,8 +256,8 @@
 asynchronously, but neither of these things is inconsistent with the Learner API.
 
 
-TODO
-~~~~
+TODO - Experiment API?
+~~~~~~~~~~~~~~~~~~~~~~
 
 I feel like something is missing from the API - and that is an interface to the graph structure
 discussed above.  The nodes in this graph are natural places to store meta-information for
@@ -266,10 +266,24 @@
 not good to say that the Learner instance *is* the node because (a) learner instances change
 during graph exploration and (b) learner instances are big, and we don't want to have to keep a
 whole saved model just to attach meta-info e.g. validation score.    Choosing this API spills
-over into other committees, so we should get their feedback about how to resolve it.
+over into other committees, so we should get their feedback about how to resolve
+it.  Maybe we need an 'Experiment' API to stand for this graph?
+
+
+TODO: Validation & Monitoring Costs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Comments
-~~~~~~~~
+Even if we do have the Experiment API as a structure to hang validation and
+monitoring results, what should be the mechanism for extracting those results.
+The Learner API is not right because extracting a monitoring cost doesn't change
+the model, doesn't change the legal instructions/edges etc.  Maybe we should use
+a similar mechanism to Instruction, called something like Measurement?  Any node
+/ learner can report the list of instructions (for moving) and the list of
+measurements (and the cost of computing them too)
+
+
+TODO - Parameter Distributions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 YB asks: it seems to me that what we really need from "Type" is not just
 testing that a value is legal, but more practically a function that specifies the
@@ -282,6 +296,22 @@
 For that reason, I think that "Type" is not a very good name.
 How about "Prior" or "Density" or something like that?
 
+JB replies: I agree that being able to choose (and update) distributions over
+these values is important. I don't think the Type structure is the right place
+to handle it though.  The challenge is to allow those distributions to change
+for a variety of reasons - e.g. the sampling distribution on the capacity
+variables is affected by the size of the dataset, it is also affected by
+previous experience in general as well as experiments on that particular
+dataset.  I'm not sure that the 'Type' structure is right to deal with this.
+Also, even with a strategy for handling these distributions, I believe a simple
+mechanism for rejecting insane values might be useful.
+
+So how should we handle it?  Hmmm...
+
+
+Comments
+~~~~~~~~
+
 OD asks: (I hope it's ok to leave comments even though I'm not in committee... I'm
 interested to see how the learner interface is shaping up so I'll be keeping
 an eye on this file)

--- a/doc/v2_planning/optimization.txt	Thu Sep 09 11:28:24 2010 -0400
+++ b/doc/v2_planning/optimization.txt	Thu Sep 09 11:50:37 2010 -0400
@@ -39,3 +39,46 @@
 
 
 
+
+Proposal for API
+================
+
+Stick to the same style of API that we've used for SGD so far.  I think it has
+worked well.  It takes theano expressions as inputs and returns theano
+expressions as results.  The caller is responsible for building those
+expressions into a callable function that does the minimization (and other
+things too maybe).
+
+
+def stochastic_gradientbased_optimization_updates(parameters, cost=None, grads=None, **kwargs):
+   """
+   :param parameters: list or tuple of Theano variables (typically shared vars)
+       that we want to optimize iteratively algorithm.
+
+   :param cost: scalar-valued Theano variable that computes noisy estimate of
+       cost  (what are the conditions on the noise?).  The cost is ignored if
+       grads are given.
+
+   :param grads: list or tuple of Theano variables representing the gradients on
+       the corresponding parameters.  These default to tensor.grad(cost,
+       parameters).
+
+   :param kwargs: algorithm-dependent arguments
+
+   :returns: a list of pairs (v, new_v) that indicate the value (new_v) each
+      variable (v) should take in order to carry out the optimization procedure.
+
+      The first section of the return value list corresponds to the terms in
+      `parameters`, and the optimization algorithm can return additional update
+      expression afterward.  This list of pairs can be passed directly to the
+      dict() constructor to create a dictionary such that dct[v] == new_v.
+   """
+
+
+Why not a class interface with an __init__ that takes the kwargs, and an
+updates() that returns the updates?  It would be wrong for auxiliary shared
+variables to be involved in two updates, so the interface should not encourage
+separate methods for those two steps.
+
+
+