annotate test_dataset.py @ 259:621faba17c60

created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
author Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
date Tue, 03 Jun 2008 16:41:55 -0400
parents bf0a1ebc6e52
children 792f81d65f82
rev   line source
51
59757365a057 the script can be autorun
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
1 #!/bin/env python
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
2 from dataset import *
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
3 from math import *
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
4 import numpy
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
5 from misc import *
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
6
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
7 def have_raised(to_eval, **var):
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
8 have_thrown = False
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
9 try:
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
10 eval(to_eval)
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
11 except :
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
12 have_thrown = True
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
13 return have_thrown
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
14
173
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
15 def have_raised2(f, *args, **kwargs):
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
16 have_thrown = False
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
17 try:
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
18 f(*args, **kwargs)
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
19 except :
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
20 have_thrown = True
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
21 return have_thrown
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
22
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
23 def test1():
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
24 print "test1"
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
25 global a,ds
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
26 a = numpy.random.rand(10,4)
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
27 print a
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
28 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
29 print "len(ds)=",len(ds)
54
70147d00615a added assert
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 52
diff changeset
30 assert(len(ds)==10)
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
31 print "example 0 = ",ds[0]
54
70147d00615a added assert
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 52
diff changeset
32 # assert
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
33 print "x=",ds["x"]
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
34 print "x|y"
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
35 for x,y in ds("x","y"):
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
36 print x,y
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
37 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4)
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
38 minibatch = minibatch_iterator.__iter__().next()
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
39 print "minibatch=",minibatch
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
40 for var in minibatch:
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
41 print "var=",var
48
b6730f9a336d Fixing MinibatchDataSet getitem
bengioy@grenat.iro.umontreal.ca
parents: 45
diff changeset
42 print "take a slice and look at field y",ds[1:6:2]["y"]
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
43
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
44 del a,ds,x,y,minibatch_iterator,minibatch,var
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
45
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
46 def test_iterate_over_examples(array,ds):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
47 #not in doc!!!
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
48 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
49 for example in range(len(ds)):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
50 assert (ds[example]['x']==array[example][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
51 assert ds[example]['y']==array[example][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
52 assert (ds[example]['z']==array[example][[0,2]]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
53 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
54 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
55 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
56
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
57 # - for example in dataset:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
58 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
59 for example in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
60 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
61 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
62 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
63 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
64 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
65 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
66 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
67 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
68
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
69 # - for val1,val2,... in dataset:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
70 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
71 for x,y,z in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
72 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
73 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
74 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
75 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
76 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
77 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
78 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
79
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
80 # - for example in dataset(field1, field2,field3, ...):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
81 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
82 for example in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
83 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
84 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
85 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
86 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
87 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
88 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
89 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
90 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
91 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
92 for example in ds('y','x'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
93 assert len(example)==2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
94 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
95 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
96 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
97 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
98 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
99 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
100
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
101 # - for val1,val2,val3 in dataset(field1, field2,field3):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
102 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
103 for x,y,z in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
104 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
105 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
106 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
107 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
108 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
109 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
110 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
111 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
112 for y,x in ds('y','x',):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
113 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
114 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
115 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
116 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
117 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
118 del x,y,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
119
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
120 def test_minibatch_size(minibatch,minibatch_size,len_ds,nb_field,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
121 ##full minibatch or the last minibatch
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
122 for idx in range(nb_field):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
123 test_minibatch_field_size(minibatch[idx],minibatch_size,len_ds,nb_iter_finished)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
124 del idx
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
125 def test_minibatch_field_size(minibatch_field,minibatch_size,len_ds,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
126 assert len(minibatch_field)==minibatch_size or ((nb_iter_finished*minibatch_size+len(minibatch_field))==len_ds and len(minibatch_field)<minibatch_size)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
127
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
128 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
129 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
130 mi=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
131 m=ds.minibatches(['x','z'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
132 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
133 for minibatch in m:
161
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
134 assert isinstance(minibatch,DataSetFields)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
135 assert len(minibatch)==2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
136 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
161
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
137 if type(ds)==ArrayDataSet:
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
138 assert (minibatch[0][:,::2]==minibatch[1]).all()
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
139 else:
162
45427d4d64b3 bugfix in test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 161
diff changeset
140 for j in xrange(len(minibatch[0])):
45427d4d64b3 bugfix in test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 161
diff changeset
141 (minibatch[0][j][::2]==minibatch[1][j]).all()
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
142 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
143 i+=len(minibatch[0])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
144 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
145 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
146 del minibatch,i,m,mi
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
147
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
148 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
149 mi=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
150 m=ds.minibatches(['x','y'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
151 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
152 for minibatch in m:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
153 assert len(minibatch)==2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
154 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
155 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
156 for id in range(len(minibatch[0])):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
157 assert (numpy.append(minibatch[0][id],minibatch[1][id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
158 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
159 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
160 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
161 del minibatch,i,id,m,mi
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
162
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
163 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
164 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
165 mi=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
166 m=ds.minibatches(['x','z'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
167 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
168 for x,z in m:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
169 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
170 test_minibatch_field_size(z,m.minibatch_size,len(ds),mi)
164
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
171 for id in range(len(x)):
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
172 assert (x[id][::2]==z[id]).all()
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
173 i+=1
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
174 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
175 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
176 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
177 del x,z,i,m,mi
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
178 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
179 mi=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
180 m=ds.minibatches(['x','y'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
181 for x,y in m:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
182 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
183 test_minibatch_field_size(y,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
184 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
185 for id in range(len(x)):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
186 assert (numpy.append(x[id],y[id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
187 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
188 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
189 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
190 del x,y,i,id,m,mi
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
191
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
192 #not in doc
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
193 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
194 m=ds.minibatches(['x','y'],n_batches=1,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
195 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
196 for x,y in m:
239
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
197 assert len(x)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
198 assert len(y)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
199 for id in range(m.minibatch_size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
200 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
201 i+=1
239
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
202 assert i==m.n_batches*m.minibatch_size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
203 del x,y,i,id,m
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
204
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
205 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
206 m=ds.minibatches(['x','y'],n_batches=2,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
207 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
208 for x,y in m:
239
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
209 assert len(x)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
210 assert len(y)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
211 for id in range(m.minibatch_size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
212 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
213 i+=1
239
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
214 assert i==m.n_batches*m.minibatch_size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
215 del x,y,i,id,m
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
216
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
217 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
218 m=ds.minibatches(['x','y'],n_batches=20,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
219 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
220 for x,y in m:
239
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
221 assert len(x)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
222 assert len(y)==m.minibatch_size
77b362a23f8e more general test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 236
diff changeset
223 for id in range(m.minibatch_size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
224 assert (numpy.append(x[id],y[id])==array[(i+4)%array.shape[0]]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
225 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
226 assert i==m.n_batches*m.minibatch_size
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
227 del x,y,i,id
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
228
240
97f35d586727 more test. Are we wanting to limit the size of minibatch to the size of the dataset?
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 239
diff changeset
229 #@todo: we can't do minibatch bigger then the size of the dataset???
97f35d586727 more test. Are we wanting to limit the size of minibatch to the size of the dataset?
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 239
diff changeset
230 assert have_raised2(ds.minibatches,['x','y'],n_batches=1,minibatch_size=len(array)+1,offset=0)
97f35d586727 more test. Are we wanting to limit the size of minibatch to the size of the dataset?
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 239
diff changeset
231 assert not have_raised2(ds.minibatches,['x','y'],n_batches=1,minibatch_size=len(array),offset=0)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
232
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
233 def test_ds_iterator(array,iterator1,iterator2,iterator3):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
234 l=len(iterator1)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
235 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
236 for x,y in iterator1:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
237 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
238 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
239 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
240 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
241 assert i==l
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
242 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
243 for y,z in iterator2:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
244 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
245 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
246 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
247 assert i==l
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
248 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
249 for x,y,z in iterator3:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
250 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
251 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
252 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
253 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
254 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
255 assert i==l
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
256
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
257 def test_getitem(array,ds):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
258 def test_ds(orig,ds,index):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
259 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
260 assert len(ds)==len(index)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
261 for x,z,y in ds('x','z','y'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
262 assert (orig[index[i]]['x']==array[index[i]][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
263 assert (orig[index[i]]['x']==x).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
264 assert orig[index[i]]['y']==array[index[i]][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
265 assert orig[index[i]]['y']==y
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
266 assert (orig[index[i]]['z']==array[index[i]][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
267 assert (orig[index[i]]['z']==z).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
268 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
269 del i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
270 ds[0]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
271 if len(ds)>2:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
272 ds[:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
273 ds[1:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
274 ds[1:1:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
275 if len(ds)>5:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
276 ds[[1,2,3]]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
277 for x in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
278 pass
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
279
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
280 #ds[:n] returns a dataset with the n first examples.
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
281 ds2=ds[:3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
282 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
283 test_ds(ds,ds2,index=[0,1,2])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
284 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
285
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
286 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s.
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
287 ds2=ds[1:7:2]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
288 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
289 test_ds(ds,ds2,[1,3,5])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
290 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
291
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
292 #ds[i]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
293 ds2=ds[5]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
294 assert isinstance(ds2,Example)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
295 assert have_raised("var['ds']["+str(len(ds))+"]",ds=ds) # index not defined
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
296 assert not have_raised("var['ds']["+str(len(ds)-1)+"]",ds=ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
297 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
298
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
299 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in.
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
300 ds2=ds[[4,7,2,8]]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
301 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
302 test_ds(ds,ds2,[4,7,2,8])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
303 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
304
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
305 #ds[fieldname]# an iterable over the values of the field fieldname across
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
306 #the ds (the iterable is obtained by default by calling valuesVStack
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
307 #over the values for individual examples).
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
308 assert have_raised("ds['h']") # h is not defined...
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
309 assert have_raised("ds[['x']]") # bad syntax
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
310 assert not have_raised("var['ds']['x']",ds=ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
311 isinstance(ds['x'],DataSetFields)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
312 ds2=ds['x']
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
313 assert len(ds['x'])==10
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
314 assert len(ds['y'])==10
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
315 assert len(ds['z'])==10
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
316 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
317 for example in ds['x']:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
318 assert (example==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
319 i+=1
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
320 assert i==len(ds)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
321 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
322 for example in ds['y']:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
323 assert (example==array[i][3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
324 i+=1
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
325 assert i==len(ds)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
326 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
327 for example in ds['z']:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
328 assert (example==array[i,0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
329 i+=1
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
330 assert i==len(ds)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
331 del ds2,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
332
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
333 #ds.<property># returns the value of a property associated with
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
334 #the name <property>. The following properties should be supported:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
335 # - 'description': a textual description or name for the ds
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
336 # - 'fieldtypes': a list of types (one per field)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
337
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
338 #* ds1 | ds2 | ds3 == ds.hstack([ds1,ds2,ds3])#????
173
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
339 #assert hstack([ds('x','y'),ds('z')])==ds
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
340 #hstack([ds('z','y'),ds('x')])==ds
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
341 assert have_raised2(hstack,[ds('x'),ds('x')])
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
342 assert have_raised2(hstack,[ds('y','x'),ds('x')])
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
343 assert not have_raised2(hstack,[ds('x'),ds('y')])
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
344
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
345 # i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
346 # for example in hstack([ds('x'),ds('y'),ds('z')]):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
347 # example==ds[i]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
348 # i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
349 # del i,example
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
350 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3])#????
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
351
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
352 def test_fields_fct(ds):
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
353 #@todo, fill correctly
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
354 assert len(ds.fields())==3
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
355 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
356 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
357 for field in ds.fields():
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
358 for field_value in field: # iterate over the values associated to that field for all the ds examples
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
359 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
360 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
361 assert i==3
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
362 assert v==3*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
363 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
364
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
365 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
366 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
367 for field in ds('x','z').fields():
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
368 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
369 for val in field:
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
370 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
371 assert i==2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
372 assert v==2*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
373 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
374
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
375 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
376 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
377 for field in ds.fields('x','y'):
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
378 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
379 for val in field:
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
380 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
381 assert i==2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
382 assert v==2*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
383 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
384
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
385 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
386 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
387 for field_examples in ds.fields():
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
388 for example_value in field_examples:
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
389 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
390 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
391 assert i==3
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
392 assert v==3*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
393 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
394
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
395 assert ds == ds.fields().examples()
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
396 assert len(ds('x','y').fields()) == 2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
397 assert len(ds('x','z').fields()) == 2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
398 assert len(ds('y').fields()) == 1
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
399
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
400 del field
194
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
401 def test_all(array,ds):
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
402 assert len(ds)==10
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
403
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
404 test_iterate_over_examples(array, ds)
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
405 test_getitem(array, ds)
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
406 test_ds_iterator(array,ds('x','y'),ds('y','z'),ds('x','y','z'))
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
407 test_fields_fct(ds)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
408
58
17729d7104fa added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 54
diff changeset
409 def test_ArrayDataSet():
17729d7104fa added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 54
diff changeset
410 #don't test stream
17729d7104fa added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 54
diff changeset
411 #tested only with float value
106
cf9bdb1d9656 added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 104
diff changeset
412 #don't always test with y
cf9bdb1d9656 added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 104
diff changeset
413 #don't test missing value
cf9bdb1d9656 added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 104
diff changeset
414 #don't test with tuple
cf9bdb1d9656 added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 104
diff changeset
415 #don't test proterties
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
416 print "test_ArrayDataSet"
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
417 a2 = numpy.random.rand(10,4)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
418 ds = ArrayDataSet(a2,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
419 ds = ArrayDataSet(a2,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
58
17729d7104fa added function test_ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 54
diff changeset
420 #assert ds==a? should this work?
100
574f4db76022 restructuring and added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 96
diff changeset
421
194
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
422 test_all(a2,ds)
169
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
423
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
424 del a2, ds
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
425
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
426 def test_LookupList():
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
427 #test only the example in the doc???
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
428 print "test_LookupList"
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
429 example = LookupList(['x','y','z'],[1,2,3])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
430 example['x'] = [1, 2, 3] # set or change a field
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
431 x, y, z = example
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
432 x = example[0]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
433 x = example["x"]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
434 assert example.keys()==['x','y','z']
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
435 assert example.values()==[[1,2,3],2,3]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
436 assert example.items()==[('x',[1,2,3]),('y',2),('z',3)]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
437 example.append_keyval('u',0) # adds item with name 'u' and value 0
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
438 assert len(example)==4 # number of items = 4 here
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
439 example2 = LookupList(['v','w'], ['a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
440 example3 = LookupList(['x','y','z','u','v','w'], [[1, 2, 3],2,3,0,'a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
441 assert example+example2==example3
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
442 assert have_raised("var['x']+var['x']",x=example)
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
443
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
444 del example, example2, example3, x, y ,z
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
445
158
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 157
diff changeset
446 def test_CachedDataSet():
148
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
447 print "test_CacheDataSet"
169
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
448 a = numpy.random.rand(10,4)
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
449 ds1 = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
148
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
450 ds2 = CachedDataSet(ds1)
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
451 ds3 = CachedDataSet(ds1,cache_all_upon_construction=True)
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
452
195
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 194
diff changeset
453 test_all(a,ds2)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 194
diff changeset
454 test_all(a,ds3)
169
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
455
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
456 del a,ds1,ds2,ds3
148
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
457
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
458
161
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
459 def test_DataSetFields():
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
460 print "test_DataSetFields"
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
461 raise NotImplementedError()
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
462
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
463 def test_ApplyFunctionDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
464 print "test_ApplyFunctionDataSet"
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
465 a = numpy.random.rand(10,4)
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
466 a2 = a+1
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
467 ds1 = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
468
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
469 ds2 = ApplyFunctionDataSet(ds1,lambda x,y,z: (x+1,y+1,z+1), ['x','y','z'],minibatch_mode=False)
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
470 ds3 = ApplyFunctionDataSet(ds1,lambda x,y,z: (numpy.array(x)+1,numpy.array(y)+1,numpy.array(z)+1),
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
471 ['x','y','z'],
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
472 minibatch_mode=True)
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
473
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
474 test_all(a2,ds2)
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
475 test_all(a2,ds3)
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
476
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
477 del a,ds1,ds2,ds3
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
478
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
479 def test_FieldsSubsetDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
480 print "test_FieldsSubsetDataSet"
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
481 raise NotImplementedError()
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
482 def test_MinibatchDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
483 print "test_MinibatchDataSet"
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
484 raise NotImplementedError()
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
485 def test_HStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
486 print "test_HStackedDataSet"
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
487 raise NotImplementedError()
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
488 def test_VStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
489 print "test_VStackedDataSet"
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
490 raise NotImplementedError()
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
491 def test_ArrayFieldsDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
492 print "test_ArrayFieldsDataSet"
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
493 raise NotImplementedError()
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
494
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
495
250
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
496 def test_speed(array, ds):
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
497 print "test_speed", ds.__class__
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
498
230
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
499 mat = numpy.random.rand(400,100)
250
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
500
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
501 @print_timing
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
502 def f_array_full(a):
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
503 a+1
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
504 @print_timing
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
505 def f_array_index(a):
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
506 for id in range(a.shape[0]):
230
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
507 # pass
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
508 a[id]+1
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
509 # a[id]*mat
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
510 @print_timing
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
511 def f_array_iter(a):
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
512 for r in a:
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
513 # pass
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
514 r+1
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
515 # r*mat
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
516 @print_timing
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
517 def f_ds_index(ds):
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
518 for id in range(len(ds)):
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
519 # pass
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
520 ds[id][0]+1
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
521 # ds[id][0]*mat
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
522 @print_timing
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
523 def f_ds_iter(ds):
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
524 for ex in ds:
230
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
525 # pass
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
526 ex[0]+1
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
527 # a[0]*mat
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
528 @print_timing
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
529 def f_ds_mb1(ds,mb_size):
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
530 for exs in ds.minibatches(minibatch_size = mb_size):
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
531 for ex in exs:
230
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
532 # pass
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
533 ex[0]+1
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
534 # ex[0]*mat
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
535 @print_timing
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
536 def f_ds_mb2(ds,mb_size):
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
537 for exs in ds.minibatches(minibatch_size = mb_size):
230
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
538 # pass
4d1bd2513e06 more test speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 229
diff changeset
539 exs[0]+1
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
540 # ex[0]*mat
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
541
250
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
542 f_array_full(array)
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
543 f_array_index(array)
474be3d12c23 make test_speed generic
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 246
diff changeset
544 f_array_iter(array)
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
545
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
546 f_ds_index(ds)
255
bf0a1ebc6e52 need to make multiple test do to cached dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 250
diff changeset
547 f_ds_index(ds)
bf0a1ebc6e52 need to make multiple test do to cached dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 250
diff changeset
548 f_ds_iter(ds)
246
0fb75fdd727d better function name in test_speed
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 240
diff changeset
549 f_ds_iter(ds)
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
550
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
551 f_ds_mb1(ds,10)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
552 f_ds_mb1(ds,100)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
553 f_ds_mb1(ds,1000)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
554 f_ds_mb1(ds,10000)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
555 f_ds_mb2(ds,10)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
556 f_ds_mb2(ds,100)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
557 f_ds_mb2(ds,1000)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
558 f_ds_mb2(ds,10000)
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
559
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
560
259
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
561
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
562
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
563
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
564
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
565 #****************************************************************
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
566 # dummy tests, less powerful than the previous tests, but can work with any new weird dataset.
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
567 # Basically, emphasis is put on consistency, but it never checks the actual values.
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
568 # To be used as a checklist, or a first test, when creating a new dataset
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
569
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
570 def dummytest_all(ds) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
571 """ Launches all the dummytests with a given dataset. """
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
572
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
573
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
574 def test1_basicstats(self,ds) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
575 """print basics stats on a dataset, like length"""
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
576
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
577 print 'len(ds) = ',len(ds)
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
578 print 'num fields = ', len(ds.fieldNames())
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
579 print 'types of field: ',
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
580 for k in ds.fieldNames() :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
581 print type(ds[0](k)[0]),
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
582 print ''
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
583
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
584 def dummytest1_basicstats(self,ds) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
585 """print basics stats on a dataset, like length"""
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
586
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
587 print 'len(ds) = ',len(ds)
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
588 print 'num fields = ', len(ds.fieldNames())
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
589 print 'types of field: ',
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
590 for k in ds.fieldNames() :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
591 print type(ds[0](k)[0]),
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
592 print ''
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
593
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
594 def dummytest2_slicing(self,ds) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
595 """test if slicing works properly"""
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
596 print 'testing slicing...',
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
597 sys.stdout.flush()
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
598
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
599 middle = len(ds) / 2
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
600 tenpercent = int(len(ds) * .1)
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
601 set1 = ds[:middle+tenpercent]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
602 set2 = ds[middle-tenpercent:]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
603 for k in range(tenpercent + tenpercent -1):
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
604 for k2 in ds.fieldNames() :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
605 if type(set1[middle-tenpercent+k](k2)[0]) == N.ndarray :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
606 for k3 in range(len(set1[middle-tenpercent+k](k2)[0])) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
607 assert set1[middle-tenpercent+k](k2)[0][k3] == set2[k](k2)[0][k3]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
608 else :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
609 assert set1[middle-tenpercent+k](k2)[0] == set2[k](k2)[0]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
610 assert tenpercent > 1
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
611 set3 = ds[middle-tenpercent:middle+tenpercent:2]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
612 for k2 in ds.fieldNames() :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
613 if type(set2[2](k2)[0]) == N.ndarray :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
614 for k3 in range(len(set2[2](k2)[0])) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
615 assert set2[2](k2)[0][k3] == set3[1](k2)[0][k3]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
616 else :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
617 assert set2[2](k2)[0] == set3[1](k2)[0]
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
618
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
619 print 'done'
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
620
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
621
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
622 def dummytest3_fields_iterator_consistency(self,ds) :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
623 """ check if the number of iterator corresponds to the number of fields"""
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
624 print 'testing fields/iterator consistency...',
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
625 sys.stdout.flush()
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
626
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
627 # basic test
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
628 maxsize = min(len(ds)-1,100)
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
629 for iter in ds[:maxsize] :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
630 assert len(iter) == len(ds.fieldNames())
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
631 if len(ds.fieldNames()) == 1 :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
632 print 'done'
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
633 return
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
634
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
635 # with minibatches iterator
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
636 ds2 = ds[:maxsize].minibatches([ds.fieldNames()[0],ds.fieldNames()[1]],minibatch_size=2)
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
637 for iter in ds2 :
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
638 assert len(iter) == 2
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
639
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
640 print 'done'
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
641
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
642
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
643
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
644
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
645
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
646
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
647
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
648
621faba17c60 created 'dummytests', tests that checks consistency of new weird datasets, where we can't compare with actual values in a matrix, for instance. Useful as a first debugging when creating a dataset
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 255
diff changeset
649
157
e8c8022582f4 only exec the test if we are the main
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 149
diff changeset
650 if __name__=='__main__':
e8c8022582f4 only exec the test if we are the main
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 149
diff changeset
651 test1()
e8c8022582f4 only exec the test if we are the main
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 149
diff changeset
652 test_LookupList()
e8c8022582f4 only exec the test if we are the main
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 149
diff changeset
653 test_ArrayDataSet()
158
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 157
diff changeset
654 test_CachedDataSet()
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
655 test_ApplyFunctionDataSet()
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
656 #test_speed()
122
d3c72e412065 removed datat that should not be commited
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 116
diff changeset
657 #test pmat.py
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
658