annotate _test_dataset.py @ 502:17945defd813

Bug fix
author Joseph Turian <turian@gmail.com>
date Wed, 29 Oct 2008 02:08:56 -0400
parents 18702ceb2096
children 82da179d95b2
rev   line source
51
59757365a057 the script can be autorun
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
1 #!/bin/env python
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
2 from dataset import *
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
3 from math import *
289
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
4 import numpy, unittest, sys
356
18702ceb2096 Added more functions
Joseph Turian <turian@iro.umontreal.ca>
parents: 352
diff changeset
5 #from misc import *
298
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
6 from lookup_list import LookupList
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
7
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
8 def have_raised(to_eval, **var):
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
9 have_thrown = False
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
10 try:
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
11 eval(to_eval)
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
12 except :
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
13 have_thrown = True
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
14 return have_thrown
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
15
173
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
16 def have_raised2(f, *args, **kwargs):
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
17 have_thrown = False
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
18 try:
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
19 f(*args, **kwargs)
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
20 except :
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
21 have_thrown = True
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
22 return have_thrown
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
23
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
24 def test1():
84
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 82
diff changeset
25 print "test1"
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
26 global a,ds
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
27 a = numpy.random.rand(10,4)
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
28 print a
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
29 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
30 print "len(ds)=",len(ds)
54
70147d00615a added assert
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 52
diff changeset
31 assert(len(ds)==10)
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
32 print "example 0 = ",ds[0]
54
70147d00615a added assert
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 52
diff changeset
33 # assert
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
34 print "x=",ds["x"]
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
35 print "x|y"
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
36 for x,y in ds("x","y"):
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
37 print x,y
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
38 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4)
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
39 minibatch = minibatch_iterator.__iter__().next()
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
40 print "minibatch=",minibatch
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
41 for var in minibatch:
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
42 print "var=",var
48
b6730f9a336d Fixing MinibatchDataSet getitem
bengioy@grenat.iro.umontreal.ca
parents: 45
diff changeset
43 print "take a slice and look at field y",ds[1:6:2]["y"]
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents:
diff changeset
44
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
45 del a,ds,x,y,minibatch_iterator,minibatch,var
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
46
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
47 def test_iterate_over_examples(array,ds):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
48 #not in doc!!!
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
49 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
50 for example in range(len(ds)):
293
4bfdda107a17 still merging
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 292
diff changeset
51 wanted = array[example][:3]
4bfdda107a17 still merging
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 292
diff changeset
52 returned = ds[example]['x']
4bfdda107a17 still merging
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 292
diff changeset
53 if (wanted != returned).all():
4bfdda107a17 still merging
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 292
diff changeset
54 print 'returned:', returned
4bfdda107a17 still merging
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 292
diff changeset
55 print 'wanted:', wanted
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
56 assert (ds[example]['x']==array[example][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
57 assert ds[example]['y']==array[example][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
58 assert (ds[example]['z']==array[example][[0,2]]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
59 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
60 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
61 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
62
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
63 # - for example in dataset:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
64 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
65 for example in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
66 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
67 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
68 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
69 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
70 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
71 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
72 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
73 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
74
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
75 # - for val1,val2,... in dataset:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
76 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
77 for x,y,z in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
78 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
79 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
80 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
81 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
82 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
83 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
84 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
85
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
86 # - for example in dataset(field1, field2,field3, ...):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
87 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
88 for example in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
89 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
90 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
91 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
92 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
93 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
94 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
95 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
96 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
97 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
98 for example in ds('y','x'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
99 assert len(example)==2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
100 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
101 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
102 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
103 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
104 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
105 del example,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
106
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
107 # - for val1,val2,val3 in dataset(field1, field2,field3):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
108 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
109 for x,y,z in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
110 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
111 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
112 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
113 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
114 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
115 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
116 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
117 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
118 for y,x in ds('y','x',):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
119 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
120 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
121 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
122 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
123 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
124 del x,y,i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
125
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
126 def test_minibatch_size(minibatch,minibatch_size,len_ds,nb_field,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
127 ##full minibatch or the last minibatch
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
128 for idx in range(nb_field):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
129 test_minibatch_field_size(minibatch[idx],minibatch_size,len_ds,nb_iter_finished)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
130 del idx
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
131 def test_minibatch_field_size(minibatch_field,minibatch_size,len_ds,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
132 assert len(minibatch_field)==minibatch_size or ((nb_iter_finished*minibatch_size+len(minibatch_field))==len_ds and len(minibatch_field)<minibatch_size)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
133
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
134 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
135 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
136 mi=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
137 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
138 m=ds.minibatches(['x','z'], minibatch_size=size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
139 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
140 for minibatch in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
141 assert isinstance(minibatch,LookupList)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
142 assert len(minibatch)==2
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
143 test_minibatch_size(minibatch,size,len(ds),2,mi)
161
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
144 if type(ds)==ArrayDataSet:
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
145 assert (minibatch[0][:,::2]==minibatch[1]).all()
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
146 else:
162
45427d4d64b3 bugfix in test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 161
diff changeset
147 for j in xrange(len(minibatch[0])):
45427d4d64b3 bugfix in test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 161
diff changeset
148 (minibatch[0][j][::2]==minibatch[1][j]).all()
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
149 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
150 i+=len(minibatch[0])
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
151 assert i==(len(ds)/size)*size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
152 assert mi==(len(ds)/size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
153 del minibatch,i,m,mi,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
154
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
155 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
156 mi=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
157 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
158 m=ds.minibatches(['x','y'], minibatch_size=size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
159 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
160 for minibatch in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
161 assert isinstance(minibatch,LookupList)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
162 assert len(minibatch)==2
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
163 test_minibatch_size(minibatch,size,len(ds),2,mi)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
164 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
165 for id in range(len(minibatch[0])):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
166 assert (numpy.append(minibatch[0][id],minibatch[1][id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
167 i+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
168 assert i==(len(ds)/size)*size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
169 assert mi==(len(ds)/size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
170 del minibatch,i,id,m,mi,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
171
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
172 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
173 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
174 mi=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
175 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
176 m=ds.minibatches(['x','z'], minibatch_size=size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
177 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
178 for x,z in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
179 test_minibatch_field_size(x,size,len(ds),mi)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
180 test_minibatch_field_size(z,size,len(ds),mi)
164
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
181 for id in range(len(x)):
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
182 assert (x[id][::2]==z[id]).all()
3518710e16ec don't assume we have an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 162
diff changeset
183 i+=1
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
184 mi+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
185 assert i==(len(ds)/size)*size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
186 assert mi==(len(ds)/size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
187 del x,z,i,m,mi,size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
188
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
189 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
190 mi=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
191 size=3
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
192 m=ds.minibatches(['x','y'], minibatch_size=3)
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
193 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
194 for x,y in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
195 assert len(x)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
196 assert len(y)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
197 test_minibatch_field_size(x,size,len(ds),mi)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
198 test_minibatch_field_size(y,size,len(ds),mi)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
199 mi+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
200 for id in range(len(x)):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
201 assert (numpy.append(x[id],y[id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
202 i+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
203 assert i==(len(ds)/size)*size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
204 assert mi==(len(ds)/size)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
205 del x,y,i,id,m,mi,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
206
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
207 #not in doc
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
208 i=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
209 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
210 m=ds.minibatches(['x','y'],n_batches=1,minibatch_size=size,offset=4)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
211 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
212 for x,y in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
213 assert len(x)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
214 assert len(y)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
215 for id in range(size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
216 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
217 i+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
218 assert i==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
219 del x,y,i,id,m,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
220
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
221 i=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
222 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
223 m=ds.minibatches(['x','y'],n_batches=2,minibatch_size=size,offset=4)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
224 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
225 for x,y in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
226 assert len(x)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
227 assert len(y)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
228 for id in range(size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
229 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
230 i+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
231 assert i==2*size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
232 del x,y,i,id,m,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
233
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
234 i=0
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
235 size=3
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
236 m=ds.minibatches(['x','y'],n_batches=20,minibatch_size=size,offset=4)
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
237 assert hasattr(m,'__iter__')
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
238 for x,y in m:
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
239 assert len(x)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
240 assert len(y)==size
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
241 for id in range(size):
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
242 assert (numpy.append(x[id],y[id])==array[(i+4)%array.shape[0]]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
243 i+=1
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
244 assert i==2*size # should not wrap
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
245 del x,y,i,id,size
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
246
341
9c08e3af975e corrected test for dataset.minibatches()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 315
diff changeset
247 assert have_raised2(ds.minibatches,['x','y'],n_batches=1,minibatch_size=len(array)+1,offset=0)
240
97f35d586727 more test. Are we wanting to limit the size of minibatch to the size of the dataset?
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 239
diff changeset
248 assert not have_raised2(ds.minibatches,['x','y'],n_batches=1,minibatch_size=len(array),offset=0)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
249
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
250 def test_ds_iterator(array,iterator1,iterator2,iterator3):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
251 l=len(iterator1)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
252 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
253 for x,y in iterator1:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
254 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
255 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
256 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
257 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
258 assert i==l
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
259 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
260 for y,z in iterator2:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
261 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
262 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
263 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
264 assert i==l
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
265 i=0
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
266 for x,y,z in iterator3:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
267 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
268 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
269 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
270 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
271 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
272 assert i==l
311
0690de82c338 a lot of tests are broken because of the new behaviour of __getitem__ that always returns a LookupList, working on that...
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 299
diff changeset
273
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
274 def test_getitem(array,ds):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
275 def test_ds(orig,ds,index):
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
276 i=0
342
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
277 assert isinstance(ds,LookupList)
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
278 assert len(ds)==3
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
279 assert len(ds[0])==len(index)
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
280 # for x,z,y in ds('x','z','y'):
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
281 for idx in index:
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
282 assert (orig[idx]['x']==array[idx][:3]).all()
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
283 assert (orig[idx]['x']==ds['x'][i]).all()
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
284 assert orig[idx]['y']==array[idx][3]
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
285 assert (orig[idx]['y']==ds['y'][i]).all() # why does it crash sometimes?
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
286 assert (orig[idx]['z']==array[idx][0:3:2]).all()
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
287 assert (orig[idx]['z']==ds['z'][i]).all()
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
288 i+=1
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
289 del i
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
290 ds[0]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
291 if len(ds)>2:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
292 ds[:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
293 ds[1:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
294 ds[1:1:1]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
295 if len(ds)>5:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
296 ds[[1,2,3]]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
297 for x in ds:
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
298 pass
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
299
342
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
300 #ds[:n] returns a LookupList with the n first examples.
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
301 ds2=ds[:3]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
302 test_ds(ds,ds2,index=[0,1,2])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
303 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
304
342
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
305 #ds[i:j] returns a LookupList with examples i,i+1,...,j-1.
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
306 ds2=ds[1:3]
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
307 test_ds(ds,ds2,index=[1,2])
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
308 del ds2
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
309
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
310 #ds[i1:i2:s] returns a LookupList with the examples i1,i1+s,...i2-s.
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
311 ds2=ds[1:7:2]
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
312 test_ds(ds,ds2,[1,3,5])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
313 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
314
342
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
315 #ds[i] returns the (i+1)-th example of the dataset.
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
316 ds2=ds[5]
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
317 assert isinstance(ds2,Example)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
318 assert have_raised("var['ds']["+str(len(ds))+"]",ds=ds) # index not defined
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
319 assert not have_raised("var['ds']["+str(len(ds)-1)+"]",ds=ds)
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
320 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
321
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
322 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in.
342
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
323 ds2=ds[[4,7,2,8]]
2259f6fa4959 fix test for __getitem__
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 341
diff changeset
324 # assert isinstance(ds2,DataSet)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
325 test_ds(ds,ds2,[4,7,2,8])
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
326 del ds2
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
327
269
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
328 #ds.<property># returns the value of a property associated with
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
329 #the name <property>. The following properties should be supported:
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
330 # - 'description': a textual description or name for the ds
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
331 # - 'fieldtypes': a list of types (one per field)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
332
269
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
333 #* ds1 | ds2 | ds3 == ds.hstack([ds1,ds2,ds3])#????
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
334 #assert hstack([ds('x','y'),ds('z')])==ds
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
335 #hstack([ds('z','y'),ds('x')])==ds
173
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
336 assert have_raised2(hstack,[ds('x'),ds('x')])
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
337 assert have_raised2(hstack,[ds('y','x'),ds('x')])
1255cd86df36 more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 169
diff changeset
338 assert not have_raised2(hstack,[ds('x'),ds('y')])
269
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
339
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
340 # i=0
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
341 # for example in hstack([ds('x'),ds('y'),ds('z')]):
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
342 # example==ds[i]
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
343 # i+=1
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
344 # del i,example
fdce496c3b56 deprecating __getitem__[fieldname] syntax
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 268
diff changeset
345 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3])#????
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
346
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
347 def test_subset(array,ds):
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
348 def test_ds(orig,ds,index):
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
349 i=0
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
350 assert isinstance(ds2,DataSet)
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
351 assert len(ds)==len(index)
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
352 for x,z,y in ds('x','z','y'):
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
353 assert (orig[index[i]]['x']==array[index[i]][:3]).all()
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
354 assert (orig[index[i]]['x']==x).all()
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
355 assert orig[index[i]]['y']==array[index[i]][3]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
356 assert orig[index[i]]['y']==y
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
357 assert (orig[index[i]]['z']==array[index[i]][0:3:2]).all()
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
358 assert (orig[index[i]]['z']==z).all()
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
359 i+=1
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
360 del i
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
361 ds[0]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
362 if len(ds)>2:
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
363 ds[:1]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
364 ds[1:1]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
365 ds[1:1:1]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
366 if len(ds)>5:
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
367 ds[[1,2,3]]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
368 for x in ds:
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
369 pass
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
370
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
371 #ds[:n] returns a dataset with the n first examples.
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
372 ds2=ds.subset[:3]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
373 test_ds(ds,ds2,index=[0,1,2])
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
374 # del ds2
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
375
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
376 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s.
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
377 ds2=ds.subset[1:7:2]
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
378 test_ds(ds,ds2,[1,3,5])
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
379 # del ds2
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
380
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
381 # #ds[i]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
382 # ds2=ds.subset[5]
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
383 # assert isinstance(ds2,Example)
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
384 # assert have_raised("var['ds']["+str(len(ds))+"]",ds=ds) # index not defined
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
385 # assert not have_raised("var['ds']["+str(len(ds)-1)+"]",ds=ds)
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
386 # del ds2
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
387
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
388 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in.
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
389 ds2=ds.subset[[4,7,2,8]]
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
390 test_ds(ds,ds2,[4,7,2,8])
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
391 # del ds2
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
392
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
393 #ds.<property># returns the value of a property associated with
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
394 #the name <property>. The following properties should be supported:
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
395 # - 'description': a textual description or name for the ds
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
396 # - 'fieldtypes': a list of types (one per field)
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
397
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
398 #* ds1 | ds2 | ds3 == ds.hstack([ds1,ds2,ds3])#????
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
399 #assert hstack([ds('x','y'),ds('z')])==ds
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
400 #hstack([ds('z','y'),ds('x')])==ds
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
401 assert have_raised2(hstack,[ds('x'),ds('x')])
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
402 assert have_raised2(hstack,[ds('y','x'),ds('x')])
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
403 assert not have_raised2(hstack,[ds('x'),ds('y')])
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
404
349
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
405 # i=0
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
406 # for example in hstack([ds('x'),ds('y'),ds('z')]):
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
407 # example==ds[i]
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
408 # i+=1
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
409 # del i,example
2174fe5cc7d0 added test for ds.subset()
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 348
diff changeset
410 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3])#????
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
411
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
412 def test_fields_fct(ds):
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
413 #@todo, fill correctly
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
414 assert len(ds.fields())==3
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
415 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
416 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
417 for field in ds.fields():
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
418 for field_value in field: # iterate over the values associated to that field for all the ds examples
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
419 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
420 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
421 assert i==3
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
422 assert v==3*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
423 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
424
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
425 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
426 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
427 for field in ds('x','z').fields():
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
428 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
429 for val in field:
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
430 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
431 assert i==2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
432 assert v==2*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
433 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
434
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
435 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
436 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
437 for field in ds.fields('x','y'):
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
438 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
439 for val in field:
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
440 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
441 assert i==2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
442 assert v==2*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
443 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
444
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
445 i=0
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
446 v=0
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
447 for field_examples in ds.fields():
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
448 for example_value in field_examples:
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
449 v+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
450 i+=1
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
451 assert i==3
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
452 assert v==3*10
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
453 del i,v
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
454
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
455 assert ds == ds.fields().examples()
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
456 assert len(ds('x','y').fields()) == 2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
457 assert len(ds('x','z').fields()) == 2
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
458 assert len(ds('y').fields()) == 1
165
2a12e7437c56 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 164
diff changeset
459
168
c704a66706fe more test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 165
diff changeset
460 del field
315
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
461
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
462 def test_overrides(ds) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
463 """ Test for examples that an override __getitem__ acts as the one in DataSet """
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
464 def ndarray_list_equal(nda,l) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
465 """
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
466 Compares if a ndarray is the same as the list. Do it by converting the list into
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
467 an numpy.ndarray, if possible
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
468 """
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
469 try :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
470 l = numpy.asmatrix(l)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
471 except :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
472 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
473 return smart_equal(nda,l)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
474
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
475 def smart_equal(a1,a2) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
476 """
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
477 Handles numpy.ndarray, LookupList, and basic containers
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
478 """
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
479 if not isinstance(a1,type(a2)) and not isinstance(a2,type(a1)):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
480 #special case: matrix vs list of arrays
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
481 if isinstance(a1,numpy.ndarray) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
482 return ndarray_list_equal(a1,a2)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
483 elif isinstance(a2,numpy.ndarray) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
484 return ndarray_list_equal(a2,a1)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
485 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
486 # compares 2 numpy.ndarray
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
487 if isinstance(a1,numpy.ndarray):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
488 if len(a1.shape) != len(a2.shape):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
489 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
490 for k in range(len(a1.shape)) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
491 if a1.shape[k] != a2.shape[k]:
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
492 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
493 return (a1==a2).all()
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
494 # compares 2 lookuplists
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
495 if isinstance(a1,LookupList) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
496 if len(a1._names) != len(a2._names) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
497 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
498 for k in a1._names :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
499 if k not in a2._names :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
500 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
501 if not smart_equal(a1[k],a2[k]) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
502 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
503 return True
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
504 # compares 2 basic containers
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
505 if hasattr(a1,'__len__'):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
506 if len(a1) != len(a2) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
507 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
508 for k in range(len(a1)) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
509 if not smart_equal(a1[k],a2[k]):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
510 return False
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
511 return True
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
512 # try basic equals
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
513 return a1 is a2
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
514
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
515 def mask(ds) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
516 class TestOverride(type(ds)):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
517 def __init__(self,ds) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
518 self.ds = ds
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
519 def __getitem__(self,key) :
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
520 res1 = self.ds[key]
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
521 res2 = DataSet.__getitem__(ds,key)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
522 assert smart_equal(res1,res2)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
523 return res1
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
524 return TestOverride(ds)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
525 # test getitem
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
526 ds2 = mask(ds)
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
527 for k in range(10):
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
528 res = ds2[k]
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
529 res = ds2[1:len(ds):3]
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
530
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
531
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
532
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
533
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
534
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
535
194
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
536 def test_all(array,ds):
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
537 assert len(ds)==10
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
538 test_iterate_over_examples(array, ds)
315
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
539 test_overrides(ds)
194
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
540 test_getitem(array, ds)
343
394b4e849c1b added test for ds.subset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 342
diff changeset
541 test_subset(array, ds)
194
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
542 test_ds_iterator(array,ds('x','y'),ds('y','z'),ds('x','y','z'))
0040ba0f0666 small refactoring
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 173
diff changeset
543 test_fields_fct(ds)
145
933db7ece663 make some function global to reuse them to test other dataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 139
diff changeset
544
315
b48cf8dce2bf test to compare overriden __getitem__ implemented, tested on ArrayDataSet.__getitem__
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 312
diff changeset
545
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
546 class T_DataSet(unittest.TestCase):
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
547 def test_ArrayDataSet(self):
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
548 #don't test stream
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
549 #tested only with float value
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
550 #don't always test with y
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
551 #don't test missing value
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
552 #don't test with tuple
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
553 #don't test proterties
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
554 a2 = numpy.random.rand(10,4)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
555 ds = ArrayDataSet(a2,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested
292
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 221 284
diff changeset
556 ds = ArrayDataSet(a2,Example(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
557 #assert ds==a? should this work?
100
574f4db76022 restructuring and added test
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 96
diff changeset
558
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
559 test_all(a2,ds)
169
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
560
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
561 del a2, ds
81
4b0859606d05 Added test for ArrayDataSet and LookUpList
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 66
diff changeset
562
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
563 def test_CachedDataSet(self):
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
564 a = numpy.random.rand(10,4)
292
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 221 284
diff changeset
565 ds1 = ArrayDataSet(a,Example(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
566 ds2 = CachedDataSet(ds1)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
567 ds3 = CachedDataSet(ds1,cache_all_upon_construction=True)
148
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
568
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
569 test_all(a,ds2)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
570 test_all(a,ds3)
169
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True)
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 168
diff changeset
571
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
572 del a,ds1,ds2,ds3
148
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
573
a5329e719229 added test for CachedDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 145
diff changeset
574
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
575 def test_DataSetFields(self):
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
576 raise NotImplementedError()
161
60e00cce3492 bugfix test in case it is not an ArrayDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 158
diff changeset
577
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
578 def test_ApplyFunctionDataSet(self):
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
579 a = numpy.random.rand(10,4)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
580 a2 = a+1
292
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 221 284
diff changeset
581 ds1 = ArrayDataSet(a,Example(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
582
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
583 ds2 = ApplyFunctionDataSet(ds1,lambda x,y,z: (x+1,y+1,z+1), ['x','y','z'],minibatch_mode=False)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
584 ds3 = ApplyFunctionDataSet(ds1,lambda x,y,z: (numpy.array(x)+1,numpy.array(y)+1,numpy.array(z)+1),
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
585 ['x','y','z'],
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
586 minibatch_mode=True)
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
587
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
588 test_all(a2,ds2)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
589 test_all(a2,ds3)
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
590
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
591 del a,ds1,ds2,ds3
202
b9950ae5e54b Added test for ApplyFunctionDataset
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 195
diff changeset
592
279
2e22cc120688 passed test_dataset.py to unittest
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 278
diff changeset
593 def test_FieldsSubsetDataSet(self):
289
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
594 a = numpy.random.rand(10,4)
297
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 295
diff changeset
595 ds = ArrayDataSet(a,Example(['x','y','z','w'],[slice(3),3,[0,2],0]))
289
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
596 ds = FieldsSubsetDataSet(ds,['x','y','z'])
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
597
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
598 test_all(a,ds)
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
599
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
600 del a, ds
295
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
601
345
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
602 def test_RenamedFieldsDataSet(self):
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
603 a = numpy.random.rand(10,4)
348
952129cd55cb corrected test_RenamedFieldsDataSet. It still fail, but this should be du to bug in RenamedFieldsDataSet.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 347
diff changeset
604 ds = ArrayDataSet(a,Example(['x1','y1','z1','w1'],[slice(3),3,[0,2],0]))
952129cd55cb corrected test_RenamedFieldsDataSet. It still fail, but this should be du to bug in RenamedFieldsDataSet.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 347
diff changeset
605 ds = RenamedFieldsDataSet(ds,['x1','y1','z1'],['x','y','z'])
345
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
606
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
607 test_all(a,ds)
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
608
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
609 del a, ds
4efb503fd0da Added test for dataset/RenamedFieldsDataSet
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 315
diff changeset
610
298
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
611 def test_MinibatchDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
612 raise NotImplementedError()
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
613 def test_HStackedDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
614 raise NotImplementedError()
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
615 def test_VStackedDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
616 raise NotImplementedError()
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
617 def test_ArrayFieldsDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
618 raise NotImplementedError()
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
619
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
620
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
621 class T_Exotic1(unittest.TestCase):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
622 class DataSet(DataSet):
295
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
623 """ Dummy dataset, where one field is a ndarray of variables size. """
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
624 def __len__(self) :
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
625 return 100
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
626 def fieldNames(self) :
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
627 return 'input','target','name'
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
628 def minibatches_nowrap(self,fieldnames,minibatch_size,n_batches,offset):
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
629 class MultiLengthDataSetIterator(object):
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
630 def __init__(self,dataset,fieldnames,minibatch_size,n_batches,offset):
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
631 if fieldnames is None: fieldnames = dataset.fieldNames()
297
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 295
diff changeset
632 self.minibatch = Example(fieldnames,range(len(fieldnames)))
295
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
633 self.dataset, self.minibatch_size, self.current = dataset, minibatch_size, offset
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
634 def __iter__(self):
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
635 return self
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
636 def next(self):
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
637 for k in self.minibatch._names :
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
638 self.minibatch[k] = []
298
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
639 for ex in range(self.minibatch_size) :
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
640 if 'input' in self.minibatch._names:
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
641 self.minibatch['input'].append( numpy.array( range(self.current + 1) ) )
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
642 if 'target' in self.minibatch._names:
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
643 self.minibatch['target'].append( self.current % 2 )
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
644 if 'name' in self.minibatch._names:
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
645 self.minibatch['name'].append( str(self.current) )
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
646 self.current += 1
295
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
647 return self.minibatch
7380376816e5 started a test for datasets where one field has a variable length. Not obvious, all tests requires a matrix as a reference
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 294
diff changeset
648 return MultiLengthDataSetIterator(self,fieldnames,minibatch_size,n_batches,offset)
298
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
649
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
650 def test_ApplyFunctionDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
651 ds = T_Exotic1.DataSet()
299
eded3cb54930 small bug fixed
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 298
diff changeset
652 dsa = ApplyFunctionDataSet(ds,lambda x,y,z: (x[-1],y*10,int(z)),['input','target','name'],minibatch_mode=False) #broken!!!!!!
298
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
653 for k in range(len(dsa)):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
654 res = dsa[k]
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
655 self.failUnless(ds[k]('input')[0][-1] == res('input')[0] , 'problem in first applied function')
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
656 res = dsa[33:96:3]
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
657
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
658 def test_CachedDataSet(self):
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
659 ds = T_Exotic1.DataSet()
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
660 dsc = CachedDataSet(ds)
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
661 for k in range(len(dsc)) :
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
662 self.failUnless(numpy.all( dsc[k]('input')[0] == ds[k]('input')[0] ) , (dsc[k],ds[k]) )
5987415496df better testing of the MultiLengthDataSet, now called exotic1
Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
parents: 297
diff changeset
663 res = dsc[:]
229
d7250ee86f72 Added speed test for ArraDataSet
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 202
diff changeset
664
157
e8c8022582f4 only exec the test if we are the main
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 149
diff changeset
665 if __name__=='__main__':
352
cefa8518ff48 code cleanup
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 350
diff changeset
666 tests = []
350
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
667 debug=False
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
668 if len(sys.argv)==1:
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
669 unittest.main()
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
670 else:
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
671 assert sys.argv[1]=="--debug"
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
672 for arg in sys.argv[2:]:
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
673 tests.append(arg)
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
674 if tests:
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
675 unittest.TestSuite(map(T_DataSet, tests)).debug()
ae6605eccd09 Can give test name in debug mode. This will execute on the gived test name.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 349
diff changeset
676 else:
289
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
677 module = __import__("_test_dataset")
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
678 tests = unittest.TestLoader().loadTestsFromModule(module)
3af204aa71e5 implemented test for FieldsSubsetDataSet. if executed, can take a --debug option to be used in a debugger
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 284
diff changeset
679 tests.debug()