--- a/doc/v2_planning/learner.txt	Thu Sep 09 11:32:42 2010 -0400
+++ b/doc/v2_planning/learner.txt	Thu Sep 09 11:49:57 2010 -0400
@@ -256,8 +256,8 @@
 asynchronously, but neither of these things is inconsistent with the Learner API.
 
 
-TODO
-~~~~
+TODO - Experiment API?
+~~~~~~~~~~~~~~~~~~~~~~
 
 I feel like something is missing from the API - and that is an interface to the graph structure
 discussed above.  The nodes in this graph are natural places to store meta-information for
@@ -266,10 +266,24 @@
 not good to say that the Learner instance *is* the node because (a) learner instances change
 during graph exploration and (b) learner instances are big, and we don't want to have to keep a
 whole saved model just to attach meta-info e.g. validation score.    Choosing this API spills
-over into other committees, so we should get their feedback about how to resolve it.
+over into other committees, so we should get their feedback about how to resolve
+it.  Maybe we need an 'Experiment' API to stand for this graph?
+
+
+TODO: Validation & Monitoring Costs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Comments
-~~~~~~~~
+Even if we do have the Experiment API as a structure to hang validation and
+monitoring results, what should be the mechanism for extracting those results.
+The Learner API is not right because extracting a monitoring cost doesn't change
+the model, doesn't change the legal instructions/edges etc.  Maybe we should use
+a similar mechanism to Instruction, called something like Measurement?  Any node
+/ learner can report the list of instructions (for moving) and the list of
+measurements (and the cost of computing them too)
+
+
+TODO - Parameter Distributions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 YB asks: it seems to me that what we really need from "Type" is not just
 testing that a value is legal, but more practically a function that specifies the
@@ -282,6 +296,22 @@
 For that reason, I think that "Type" is not a very good name.
 How about "Prior" or "Density" or something like that?
 
+JB replies: I agree that being able to choose (and update) distributions over
+these values is important. I don't think the Type structure is the right place
+to handle it though.  The challenge is to allow those distributions to change
+for a variety of reasons - e.g. the sampling distribution on the capacity
+variables is affected by the size of the dataset, it is also affected by
+previous experience in general as well as experiments on that particular
+dataset.  I'm not sure that the 'Type' structure is right to deal with this.
+Also, even with a strategy for handling these distributions, I believe a simple
+mechanism for rejecting insane values might be useful.
+
+So how should we handle it?  Hmmm...
+
+
+Comments
+~~~~~~~~
+
 OD asks: (I hope it's ok to leave comments even though I'm not in committee... I'm
 interested to see how the learner interface is shaping up so I'll be keeping
 an eye on this file)