Mercurial > pylearn
annotate test_dataset.py @ 94:9c8f3c9c247b
corrected the use of .all()
| author | Frederic Bastien <bastienf@iro.umontreal.ca> |
|---|---|
| date | Mon, 05 May 2008 17:44:49 -0400 |
| parents | eee739fefdff |
| children | 352910e0dbf5 |
| rev | line source |
|---|---|
|
51
59757365a057
the script can be autorun
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
45
diff
changeset
|
1 #!/bin/env python |
| 45 | 2 from dataset import * |
| 3 from math import * | |
| 4 import numpy | |
| 5 | |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
6 def have_raised(to_eval): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
7 have_thrown = False |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
8 try: |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
9 eval(to_eval) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
10 except : |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
11 have_thrown = True |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
12 return have_thrown |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
13 |
| 45 | 14 def test1(): |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
15 print "test1" |
| 45 | 16 global a,ds |
| 17 a = numpy.random.rand(10,4) | |
| 18 print a | |
| 19 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]}) | |
| 20 print "len(ds)=",len(ds) | |
| 54 | 21 assert(len(ds)==10) |
| 45 | 22 print "example 0 = ",ds[0] |
| 54 | 23 # assert |
| 45 | 24 print "x=",ds["x"] |
| 25 print "x|y" | |
| 26 for x,y in ds("x","y"): | |
| 27 print x,y | |
| 28 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4) | |
| 29 minibatch = minibatch_iterator.__iter__().next() | |
| 30 print "minibatch=",minibatch | |
| 31 for var in minibatch: | |
| 32 print "var=",var | |
|
48
b6730f9a336d
Fixing MinibatchDataSet getitem
bengioy@grenat.iro.umontreal.ca
parents:
45
diff
changeset
|
33 print "take a slice and look at field y",ds[1:6:2]["y"] |
| 45 | 34 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
35 def test_ArrayDataSet(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
36 #don't test stream |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
37 #tested only with float value |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
38 #test with y too |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
39 #test missing value |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
40 |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
41 print "test_ArrayDataSet" |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
42 a = numpy.random.rand(10,4) |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
43 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
44 ds = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
45 assert len(ds)==10 |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
46 #assert ds==a? should this work? |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
47 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
48 #not in doc!!! |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
49 for example in range(len(ds)): |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
50 assert (ds[example]['x']==a[example][:3]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
51 assert ds[example]['y']==a[example][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
52 assert (ds[example]['z']==a[example][[0,2]]).all() |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
53 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
54 # - for example in dataset: |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
55 i=0 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
56 for example in ds: |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
57 assert (example['x']==a[i][:3]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
58 assert example['y']==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
59 assert (example['z']==a[i][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
60 assert (numpy.append(example['x'],example['y'])==a[i]).all() |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
61 i+=1 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
62 assert i==len(ds) |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
63 # - for val1,val2,... in dataset: |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
64 i=0 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
65 for x,y,z in ds: |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
66 assert (x==a[i][:3]).all() |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
67 assert y==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
68 assert (z==a[i][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
69 assert (numpy.append(x,y)==a[i]).all() |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
70 i+=1 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
71 assert i==len(ds) |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
72 # - for example in dataset(field1, field2,field3, ...): |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
73 i=0 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
74 for example in ds('x','y','z'): |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
75 assert (example['x']==a[i][:3]).all() |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
76 assert example['y']==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
77 assert (example['z']==a[i][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
78 assert (numpy.append(example['x'],example['y'])==a[i]).all() |
|
91
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
79 i+=1 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
80 assert i==len(ds) |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
81 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
82 # - for val1,val2,val3 in dataset(field1, field2,field3): |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
83 |
|
eee739fefdff
corrected test from discution about the syntax with Yoshua
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
89
diff
changeset
|
84 # - for example in dataset(field1, field2,field3, ...): |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
85 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
86 def test_ds_iterator(iterator1,iterator2,iterator3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
87 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
88 for x,y in iterator1: |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
89 assert (x==a[i][:3]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
90 assert y==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
91 assert (numpy.append(x,y)==a[i]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
92 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
93 assert i==len(ds) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
94 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
95 for y,z in iterator2: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
96 assert y==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
97 assert (z==a[i][0:3:2]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
98 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
99 assert i==len(ds) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
100 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
101 for x,y,z in iterator3: |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
102 assert (x==a[i][:3]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
103 assert y==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
104 assert (z==a[i][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
105 assert (numpy.append(x,y)==a[i]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
106 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
107 assert i==len(ds) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
108 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
109 #not in doc!!! - for val1,val2,val3 in dataset(field1, field2,field3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
110 test_ds_iterator(ds('x','y'),ds('y','z'),ds('x','y','z')) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
111 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
112 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
113 for minibatch in ds.minibatches(['x','z'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
114 assert len(minibatch)==2 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
115 assert len(minibatch[0])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
116 assert len(minibatch[1])==3 |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
117 assert (minibatch[0][:,0:3:2]==minibatch[1]).all() |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
118 i=0 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
119 for minibatch in ds.minibatches(['x','y'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
120 assert len(minibatch)==2 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
121 assert len(minibatch[0])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
122 assert len(minibatch[1])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
123 for id in range(3): |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
124 assert (numpy.append(minibatch[0][id],minibatch[1][id])==a[i]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
125 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
126 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
127 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N): |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
128 for x,z in ds.minibatches(['x','z'], minibatch_size=3): |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
129 assert len(x)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
130 assert len(z)==3 |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
131 assert (x[:,0:3:2]==z).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
132 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
133 for x,y in ds.minibatches(['x','y'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
134 assert len(x)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
135 assert len(y)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
136 for id in range(3): |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
137 assert (numpy.append(x[id],y[id])==a[i]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
138 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
139 # - for x,y,z in dataset: # fail x,y,z order not fixed as it is a dict. |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
140 # for x,y,z in ds: |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
141 # assert (x==a[i][:2]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
142 # assert y==a[i][3] |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
143 # assert (z==a[i][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
144 # assert (numpy.append(x,y)==a[i]).all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
145 # i+=1 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
146 |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
147 # for minibatch in ds.minibatches(['z','y'], minibatch_size=3): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
148 # print minibatch |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
149 # minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4) |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
150 # minibatch = minibatch_iterator.__iter__().next() |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
151 # print "minibatch=",minibatch |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
152 # for var in minibatch: |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
153 # print "var=",var |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
154 # print "take a slice and look at field y",ds[1:6:2]["y"] |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
155 assert have_raised("ds['h']") # h is not defined... |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
156 assert have_raised("ds[['h']]") # h is not defined... |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
157 |
| 63 | 158 assert len(ds.fields())==3 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
159 for field in ds.fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
160 for field_value in field: # iterate over the values associated to that field for all the ds examples |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
161 pass |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
162 for field in ds('x','z').fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
163 pass |
| 60 | 164 for field in ds.fields('x','y'): |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
165 pass |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
166 for field_examples in ds.fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
167 for example_value in field_examples: |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
168 pass |
| 63 | 169 |
| 170 assert ds == ds.fields().examples() | |
| 171 | |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
172 def test_ds(orig,ds,index): |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
173 i=0 |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
174 assert len(ds)==len(index) |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
175 for x,z,y in ds('x','z','y'): |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
176 assert (orig[index[i]]['x']==a[index[i]][:3]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
177 assert (orig[index[i]]['x']==x).all() |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
178 assert orig[index[i]]['y']==a[index[i]][3] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
179 assert orig[index[i]]['y']==y |
|
94
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
180 assert (orig[index[i]]['z']==a[index[i]][0:3:2]).all() |
|
9c8f3c9c247b
corrected the use of .all()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
91
diff
changeset
|
181 assert (orig[index[i]]['z']==z).all() |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
182 i+=1 |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
183 del i |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
184 ds[0] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
185 if len(ds)>2: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
186 ds[:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
187 ds[1:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
188 ds[1:1:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
189 if len(ds)>5: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
190 ds[[1,2,3]] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
191 for x in ds: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
192 pass |
| 63 | 193 |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
194 #ds[:n] returns a dataset with the n first examples. |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
195 ds2=ds[:3] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
196 test_ds(ds,ds2,index=[0,1,2]) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
197 |
| 63 | 198 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s. |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
199 ds2=ds[1:7:2] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
200 test_ds(ds,ds2,[1,3,5]) |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
201 |
| 63 | 202 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in. |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
203 ds2=ds[[4,7,2,8]] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
204 test_ds(ds,ds2,[4,7,2,8]) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
205 #ds[i1,i2,...]# should we accept???? |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
206 |
| 63 | 207 #ds[fieldname]# an iterable over the values of the field fieldname across |
| 208 #the ds (the iterable is obtained by default by calling valuesVStack | |
| 209 #over the values for individual examples). | |
| 210 | |
| 211 #ds.<property># returns the value of a property associated with | |
| 212 #the name <property>. The following properties should be supported: | |
| 213 # - 'description': a textual description or name for the ds | |
| 214 # - 'fieldtypes': a list of types (one per field) | |
| 215 #* ds1 | ds2 | ds3 == ds.hstack([ds1,ds2,ds3]) | |
| 216 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3]) | |
| 217 | |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
218 # for (x,y) in (ds('x','y'),a): #don't work # haven't found a variant that work. |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
219 # assert numpy.append(x,y)==z |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
220 |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
221 def test_LookupList(): |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
222 #test only the example in the doc??? |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
223 print "test_LookupList" |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
224 example = LookupList(['x','y','z'],[1,2,3]) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
225 example['x'] = [1, 2, 3] # set or change a field |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
226 x, y, z = example |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
227 x = example[0] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
228 x = example["x"] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
229 assert example.keys()==['x','y','z'] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
230 assert example.values()==[[1,2,3],2,3] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
231 assert example.items()==[('x',[1,2,3]),('y',2),('z',3)] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
232 example.append_keyval('u',0) # adds item with name 'u' and value 0 |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
233 assert len(example)==4 # number of items = 4 here |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
234 example2 = LookupList(['v','w'], ['a','b']) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
235 example3 = LookupList(['x','y','z','u','v','w'], [[1, 2, 3],2,3,0,'a','b']) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
236 assert example+example2==example3 |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
237 assert have_raised("example+example") |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
238 |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
239 def test_ApplyFunctionDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
240 print "test_ApplyFunctionDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
241 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
242 def test_CacheDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
243 print "test_CacheDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
244 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
245 def test_FieldsSubsetDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
246 print "test_FieldsSubsetDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
247 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
248 def test_DataSetFields(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
249 print "test_DataSetFields" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
250 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
251 def test_MinibatchDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
252 print "test_MinibatchDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
253 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
254 def test_HStackedDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
255 print "test_HStackedDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
256 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
257 def test_VStackedDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
258 print "test_VStackedDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
259 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
260 def test_ArrayFieldsDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
261 print "test_ArrayFieldsDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
262 raise NotImplementedError() |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
263 |
|
74
b4159cbdc06b
Fixed errors raised by test_dataset
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
66
diff
changeset
|
264 test1() |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
265 test_LookupList() |
|
65
d48eba49a2f4
fixed the infinite loop
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
64
diff
changeset
|
266 test_ArrayDataSet() |
|
64
863da25a60f1
trying to fix infinite loop
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
63
diff
changeset
|
267 |