pylearn: doc/v2_planning/dataset.txt comparison

dataset: Added a couple points we did not have time to discuss during meeting

author	Olivier Delalleau <delallea@iro>
date	Fri, 10 Sep 2010 15:36:23 -0400
parents	5c14d2ffcbb3
children	4c00af69c164

comparison

equal deleted inserted replaced

-:446bd478953f
+:f9f72ae84313
 if we wanted to provide a virtual world for an agent as a dataset then we need
 to provide 'actions' to get the next batch.  Could this be done in the current
 API?
+Field names and attributes
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+OD: One important question is how to handle fields' names and characteristics.
+For instance, it can be useful to know that the 3rd input field represents a
+number of fingers, and is a non-negative discrete field whose numeric value is
+meaningful (compared, to, say, an integer index that would correspond to an
+animal's category). We mentioned metadata during the meeting, but we did not
+get into its details: that may be a place where to put this kind of things.
+Freeing memory
+~~~~~~~~~~~~~~
+OD: It is sometimes useful to be able to free memory used by previous
+computations. A typical example is when you load in memory the original
+dataset, then perform various processing steps, ending with a new dataset that
+you also store in memory before feeding it to the learner. Unless you very
+carefully design your code to avoid it, your original dataset will still
+remain in memory (as well as maybe the results of some computations performed
+along the way). So there may be a use for a `clear()` method that would be
+called by the topmost dataset (the one doing the final memory caching), and
+would be forwarded iteratively to previous datasets so as to get back all this
+wasted memory space.

Mercurial > pylearn