Mercurial > pylearn
annotate test_dataset.py @ 89:05dc4804357b
more test and refactoring
| author | Frederic Bastien <bastienf@iro.umontreal.ca> |
|---|---|
| date | Mon, 05 May 2008 16:54:16 -0400 |
| parents | 3fd6879e0f76 |
| children | eee739fefdff |
| rev | line source |
|---|---|
|
51
59757365a057
the script can be autorun
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
45
diff
changeset
|
1 #!/bin/env python |
| 45 | 2 from dataset import * |
| 3 from math import * | |
| 4 import numpy | |
| 5 | |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
6 def have_raised(to_eval): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
7 have_thrown = False |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
8 try: |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
9 eval(to_eval) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
10 except : |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
11 have_thrown = True |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
12 return have_thrown |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
13 |
| 45 | 14 def test1(): |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
15 print "test1" |
| 45 | 16 global a,ds |
| 17 a = numpy.random.rand(10,4) | |
| 18 print a | |
| 19 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]}) | |
| 20 print "len(ds)=",len(ds) | |
| 54 | 21 assert(len(ds)==10) |
| 45 | 22 print "example 0 = ",ds[0] |
| 54 | 23 # assert |
| 45 | 24 print "x=",ds["x"] |
| 25 print "x|y" | |
| 26 for x,y in ds("x","y"): | |
| 27 print x,y | |
| 28 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4) | |
| 29 minibatch = minibatch_iterator.__iter__().next() | |
| 30 print "minibatch=",minibatch | |
| 31 for var in minibatch: | |
| 32 print "var=",var | |
|
48
b6730f9a336d
Fixing MinibatchDataSet getitem
bengioy@grenat.iro.umontreal.ca
parents:
45
diff
changeset
|
33 print "take a slice and look at field y",ds[1:6:2]["y"] |
| 45 | 34 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
35 def test_ArrayDataSet(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
36 #don't test stream |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
37 #tested only with float value |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
38 #test with y too |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
39 #test missing value |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
40 |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
41 print "test_ArrayDataSet" |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
42 a = numpy.random.rand(10,4) |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
43 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
44 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
45 assert len(ds)==10 |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
46 #assert ds==a? should this work? |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
47 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
48 #not in doc!!! |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
49 for example in range(len(ds)): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
50 assert ds[example]['x'].all()==a[example][:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
51 assert ds[example]['y']==a[example][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
52 assert ds[example]['z'].all()==a[example][0:3:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
53 # - for example in dataset:: |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
54 i=0 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
55 for example in ds: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
56 assert example['x'].all()==a[i][:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
57 assert example['y']==a[i][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
58 assert example['z'].all()==a[i][0:3:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
59 assert numpy.append(example['x'],example['y']).all()==a[i].all() |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
60 i+=1 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
61 assert i==len(ds) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
62 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
63 def test_ds_iterator(iterator1,iterator2,iterator3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
64 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
65 for x,y in iterator1: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
66 assert x.all()==a[i][:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
67 assert y==a[i][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
68 assert numpy.append(x,y).all()==a[i].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
69 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
70 assert i==len(ds) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
71 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
72 for y,z in iterator2: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
73 assert y==a[i][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
74 assert z.all()==a[i][0:3:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
75 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
76 assert i==len(ds) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
77 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
78 for x,y,z in iterator3: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
79 assert x.all()==a[i][:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
80 assert y==a[i][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
81 assert z.all()==a[i][0:3:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
82 assert numpy.append(x,y).all()==a[i].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
83 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
84 assert i==len(ds) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
85 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
86 #not in doc!!! - for val1,val2,val3 in dataset(field1, field2,field3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
87 test_ds_iterator(ds('x','y'),ds('y','z'),ds('x','y','z')) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
88 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
89 #not in doc!!! - for val1,val2,val3 in dataset((field1, field2,field3)): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
90 test_ds_iterator(ds(('x','y')),ds(('y','z')),ds(('x','y','z'))) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
91 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
92 # - for val1,val2,val3 in dataset([field1, field2,field3]): #was bugged |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
93 test_ds_iterator(ds(['x','y']),ds(['y','z']),ds(['x','y','z'])) |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
94 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
95 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
96 for minibatch in ds.minibatches(['x','z'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
97 assert len(minibatch)==2 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
98 assert len(minibatch[0])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
99 assert len(minibatch[1])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
100 assert minibatch[0][:,0:3:2].all()==minibatch[1].all() |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
101 i=0 |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
102 for minibatch in ds.minibatches(['x','y'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
103 assert len(minibatch)==2 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
104 assert len(minibatch[0])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
105 assert len(minibatch[1])==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
106 for id in range(3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
107 assert numpy.append(minibatch[0][id],minibatch[1][id]).all()==a[i].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
108 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
109 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
110 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N): |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
111 for x,z in ds.minibatches(['x','z'], minibatch_size=3): |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
112 assert len(x)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
113 assert len(z)==3 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
114 assert x[:,0:3:2].all()==z.all() |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
115 i=0 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
116 for x,y in ds.minibatches(['x','y'], minibatch_size=3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
117 assert len(x)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
118 assert len(y)==3 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
119 for id in range(3): |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
120 assert numpy.append(minibatch[0][id],minibatch[1][id]).all()==a[i].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
121 i+=1 |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
122 # - for x,y,z in dataset: # fail x,y,z order not fixed as it is a dict. |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
123 # for x,y,z in ds: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
124 # assert x.all()==a[i][:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
125 # assert y==a[i][3] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
126 # assert z.all()==a[i][0:3:2].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
127 # assert numpy.append(x,y).all()==a[i].all() |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
128 # i+=1 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
129 |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
130 # for minibatch in ds.minibatches(['z','y'], minibatch_size=3): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
131 # print minibatch |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
132 # minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4) |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
133 # minibatch = minibatch_iterator.__iter__().next() |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
134 # print "minibatch=",minibatch |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
135 # for var in minibatch: |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
136 # print "var=",var |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
137 # print "take a slice and look at field y",ds[1:6:2]["y"] |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
138 assert have_raised("ds['h']") # h is not defined... |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
139 assert have_raised("ds[['h']]") # h is not defined... |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
140 |
| 63 | 141 assert len(ds.fields())==3 |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
142 for field in ds.fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
143 for field_value in field: # iterate over the values associated to that field for all the ds examples |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
144 pass |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
145 for field in ds('x','z').fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
146 pass |
| 60 | 147 for field in ds.fields('x','y'): |
|
58
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
148 pass |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
149 for field_examples in ds.fields(): |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
150 for example_value in field_examples: |
|
17729d7104fa
added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
54
diff
changeset
|
151 pass |
| 63 | 152 |
| 153 assert ds == ds.fields().examples() | |
| 154 | |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
155 def test_ds(orig,ds,index): |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
156 i=0 |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
157 assert len(ds)==len(index) |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
158 for x,z,y in ds('x','z','y'): |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
159 assert orig[index[i]]['x'].all()==a[index[i]][:3].all() |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
160 assert orig[index[i]]['x'].all()==x.all() |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
161 assert orig[index[i]]['y']==a[index[i]][3] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
162 assert orig[index[i]]['y']==y |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
163 assert orig[index[i]]['z'].all()==a[index[i]][0:3:2].all() |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
164 assert orig[index[i]]['z'].all()==z.all() |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
165 i+=1 |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
166 del i |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
167 ds[0] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
168 if len(ds)>2: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
169 ds[:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
170 ds[1:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
171 ds[1:1:1] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
172 if len(ds)>5: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
173 ds[[1,2,3]] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
174 for x in ds: |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
175 pass |
| 63 | 176 |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
177 #ds[:n] returns a dataset with the n first examples. |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
178 ds2=ds[:3] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
179 test_ds(ds,ds2,index=[0,1,2]) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
180 |
| 63 | 181 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s. |
|
86
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
182 ds2=ds[1:7:2] |
|
fdf72ea4f2bc
added function test_ds in test_ArrayDataSet who test a sub dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
84
diff
changeset
|
183 test_ds(ds,ds2,[1,3,5]) |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
184 |
| 63 | 185 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in. |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
186 ds2=ds[[4,7,2,8]] |
|
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
187 test_ds(ds,ds2,[4,7,2,8]) |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
188 #ds[i1,i2,...]# should we accept???? |
|
89
05dc4804357b
more test and refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
87
diff
changeset
|
189 |
| 63 | 190 #ds[fieldname]# an iterable over the values of the field fieldname across |
| 191 #the ds (the iterable is obtained by default by calling valuesVStack | |
| 192 #over the values for individual examples). | |
| 193 | |
| 194 #ds.<property># returns the value of a property associated with | |
| 195 #the name <property>. The following properties should be supported: | |
| 196 # - 'description': a textual description or name for the ds | |
| 197 # - 'fieldtypes': a list of types (one per field) | |
| 198 #* ds1 | ds2 | ds3 == ds.hstack([ds1,ds2,ds3]) | |
| 199 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3]) | |
| 200 | |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
201 # for (x,y) in (ds('x','y'),a): #don't work # haven't found a variant that work. |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
202 # assert numpy.append(x,y)==z |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
203 |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
204 def test_LookupList(): |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
205 #test only the example in the doc??? |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
206 print "test_LookupList" |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
207 example = LookupList(['x','y','z'],[1,2,3]) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
208 example['x'] = [1, 2, 3] # set or change a field |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
209 x, y, z = example |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
210 x = example[0] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
211 x = example["x"] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
212 assert example.keys()==['x','y','z'] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
213 assert example.values()==[[1,2,3],2,3] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
214 assert example.items()==[('x',[1,2,3]),('y',2),('z',3)] |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
215 example.append_keyval('u',0) # adds item with name 'u' and value 0 |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
216 assert len(example)==4 # number of items = 4 here |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
217 example2 = LookupList(['v','w'], ['a','b']) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
218 example3 = LookupList(['x','y','z','u','v','w'], [[1, 2, 3],2,3,0,'a','b']) |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
219 assert example+example2==example3 |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
220 assert have_raised("example+example") |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
221 |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
222 def test_ApplyFunctionDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
223 print "test_ApplyFunctionDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
224 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
225 def test_CacheDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
226 print "test_CacheDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
227 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
228 def test_FieldsSubsetDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
229 print "test_FieldsSubsetDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
230 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
231 def test_DataSetFields(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
232 print "test_DataSetFields" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
233 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
234 def test_MinibatchDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
235 print "test_MinibatchDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
236 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
237 def test_HStackedDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
238 print "test_HStackedDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
239 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
240 def test_VStackedDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
241 print "test_VStackedDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
242 raise NotImplementedError() |
|
84
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
243 def test_ArrayFieldsDataSet(): |
|
aa9e786ee849
added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
82
diff
changeset
|
244 print "test_ArrayFieldsDataSet" |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
245 raise NotImplementedError() |
|
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
246 |
|
74
b4159cbdc06b
Fixed errors raised by test_dataset
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
66
diff
changeset
|
247 test1() |
|
81
4b0859606d05
Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
66
diff
changeset
|
248 test_LookupList() |
|
65
d48eba49a2f4
fixed the infinite loop
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
64
diff
changeset
|
249 test_ArrayDataSet() |
|
64
863da25a60f1
trying to fix infinite loop
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
63
diff
changeset
|
250 |