Mercurial > pylearn
annotate _test_dataset.py @ 7:6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
| author | bengioy@bengiomac.local |
|---|---|
| date | Mon, 24 Mar 2008 13:20:15 -0400 |
| parents | d5738b79089a |
| children | d1c394486037 |
| rev | line source |
|---|---|
| 4 | 1 from dataset import * |
| 2 from math import * | |
| 3 import unittest | |
| 4 | |
| 5 def _sum_all(a): | |
| 6 s=a | |
| 7 while isinstance(s,numpy.ndarray): | |
| 8 s=sum(s) | |
| 9 return s | |
| 10 | |
| 11 class T_arraydataset(unittest.TestCase): | |
| 12 def setUp(self): | |
| 13 numpy.random.seed(123456) | |
| 14 | |
| 15 def test0(self): | |
|
6
d5738b79089a
Removed MinibatchIterator and instead made minibatch_size a field of all DataSets,
bengioy@bengiomac.local
parents:
4
diff
changeset
|
16 a=ArrayDataSet(data=numpy.random.rand(8,3),fields={"x":slice(2),"y":slice(1,3)},minibatch_size=1) |
| 4 | 17 s=0 |
| 18 for example in a: | |
|
7
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
19 print len(example), example.x |
| 4 | 20 s+=_sum_all(example.x) |
| 21 print s | |
|
7
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
22 self.failUnless(abs(s-7.25967597)<1e-6) |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
23 |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
24 def test1(self): |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
25 a=ArrayDataSet(data=numpy.random.rand(10,4),fields={"x":slice(2),"y":slice(1,4)},minibatch_size=1) |
|
6
d5738b79089a
Removed MinibatchIterator and instead made minibatch_size a field of all DataSets,
bengioy@bengiomac.local
parents:
4
diff
changeset
|
26 a.minibatch_size=2 |
|
7
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
27 print a.asarray() |
|
6
d5738b79089a
Removed MinibatchIterator and instead made minibatch_size a field of all DataSets,
bengioy@bengiomac.local
parents:
4
diff
changeset
|
28 for mb in a: |
|
7
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
29 print mb,mb.asarray() |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
30 print "a.y=",a.y |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
31 for mb in ArrayDataSet(data=a.y,minibatch_size=2): |
|
6
d5738b79089a
Removed MinibatchIterator and instead made minibatch_size a field of all DataSets,
bengioy@bengiomac.local
parents:
4
diff
changeset
|
32 print mb |
|
7
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
33 for e in mb: |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
34 print e |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
35 self.failUnless(True) |
|
6f8f338686db
Moved iterating counter into a FiniteDataSetIterator to allow embedded iterations and multiple threads iterating at the same time on a dataset.
bengioy@bengiomac.local
parents:
6
diff
changeset
|
36 |
| 4 | 37 if __name__ == '__main__': |
| 38 unittest.main() | |
| 39 |