Mercurial > pylearn
comparison doc/v2_planning/dataset.txt @ 1109:29b48deb6a84
reply/comment regarding the GPU and datasets
| author | Razvan Pascanu <r.pascanu@gmail.com> |
|---|---|
| date | Tue, 14 Sep 2010 09:01:16 -0400 |
| parents | 546bd0ccb0e4 |
| children | 4797a4cb73e1 |
comparison
equal
deleted
inserted
replaced
| 1108:c5c7ba805a2f | 1109:29b48deb6a84 |
|---|---|
| 341 | 341 |
| 342 OD asks: Ok, so why would the dataset have to be responsible for providing a | 342 OD asks: Ok, so why would the dataset have to be responsible for providing a |
| 343 shared variable? Why wouldn't the learner just create this shared variable | 343 shared variable? Why wouldn't the learner just create this shared variable |
| 344 internally and copy into it the data provided by the dataset? | 344 internally and copy into it the data provided by the dataset? |
| 345 | 345 |
| 346 RP replies: Sure, the learner could take care of all this. Note though that the | |
| 347 learner should take care to divide the dataset into chunks that fit in the | |
| 348 GPU memory ( in case of a large dataset) and then take care of updating the | |
| 349 shared variables acording to the current chunk. Personally I feel like all | |
| 350 this data division, management and so on should be done by the dataset. | |
| 351 It feels more natural that way. For example assume you have a dataset that | |
| 352 is composed of a time series and some static data ( carre-tech heart beat | |
| 353 data is a good example). The static data is small enough so that you could | |
| 354 always store on the GPU, and you would only need to split the time series. | |
| 355 For the learner to do this ( since it gets the same interface from any | |
| 356 dataset object) would be like and if <this case> then, while for the | |
| 357 dataset is just a different class. But I'm happy to have all this GPU stuff | |
| 358 send to the learner as well if everybody else believe that is better. | |
| 359 |