pylearn: test_dataset.py annotate

annotate test_dataset.py @ 212:9b57ea8c767f

previous commit was supposed to concern only one file, dataset.py, try to undo my other changes with this commit (nothing was broken though, just useless debugging prints)

author	Thierry Bertin-Mahieux <bertinmt@iro.umontreal.ca>
date	Wed, 21 May 2008 17:42:20 -0400
parents	b9950ae5e54b
children	3c96d23b36ac d7250ee86f72

rev	line source
51 59757365a057 the script can be autorun Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	1 #!/bin/env python
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	2 from dataset import *
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	3 from math import *
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	4 import numpy
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	5
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	6 def have_raised(to_eval, **var):
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	7 have_thrown = False
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	8 try:
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	9 eval(to_eval)
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	10 except :
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	11 have_thrown = True
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	12 return have_thrown
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	13
173 1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	14 def have_raised2(f, args, *kwargs):
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	15 have_thrown = False
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	16 try:
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	17 f(args, *kwargs)
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	18 except :
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	19 have_thrown = True
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	20 return have_thrown
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	21
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	22 def test1():
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	23 print "test1"
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	24 global a,ds
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	25 a = numpy.random.rand(10,4)
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	26 print a
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	27 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	28 print "len(ds)=",len(ds)
54 70147d00615a added assert Frederic Bastien <bastienf@iro.umontreal.ca> parents: 52 diff changeset	29 assert(len(ds)==10)
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	30 print "example 0 = ",ds[0]
54 70147d00615a added assert Frederic Bastien <bastienf@iro.umontreal.ca> parents: 52 diff changeset	31 # assert
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	32 print "x=",ds["x"]
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	33 print "x\|y"
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	34 for x,y in ds("x","y"):
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	35 print x,y
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	36 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4)
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	37 minibatch = minibatch_iterator.__iter__().next()
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	38 print "minibatch=",minibatch
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	39 for var in minibatch:
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	40 print "var=",var
48 b6730f9a336d Fixing MinibatchDataSet getitem bengioy@grenat.iro.umontreal.ca parents: 45 diff changeset	41 print "take a slice and look at field y",ds[1:6:2]["y"]
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	42
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	43 del a,ds,x,y,minibatch_iterator,minibatch,var
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	44
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	45 def test_iterate_over_examples(array,ds):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	46 #not in doc!!!
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	47 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	48 for example in range(len(ds)):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	49 assert (ds[example]['x']==array[example][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	50 assert ds[example]['y']==array[example][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	51 assert (ds[example]['z']==array[example][[0,2]]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	52 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	53 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	54 del example,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	55
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	56 # - for example in dataset:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	57 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	58 for example in ds:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	59 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	60 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	61 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	62 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	63 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	64 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	65 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	66 del example,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	67
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	68 # - for val1,val2,... in dataset:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	69 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	70 for x,y,z in ds:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	71 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	72 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	73 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	74 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	75 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	76 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	77 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	78
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	79 # - for example in dataset(field1, field2,field3, ...):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	80 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	81 for example in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	82 assert len(example)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	83 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	84 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	85 assert (example['z']==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	86 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	87 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	88 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	89 del example,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	90 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	91 for example in ds('y','x'):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	92 assert len(example)==2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	93 assert (example['x']==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	94 assert example['y']==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	95 assert (numpy.append(example['x'],example['y'])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	96 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	97 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	98 del example,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	99
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	100 # - for val1,val2,val3 in dataset(field1, field2,field3):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	101 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	102 for x,y,z in ds('x','y','z'):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	103 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	104 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	105 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	106 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	107 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	108 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	109 del x,y,z,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	110 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	111 for y,x in ds('y','x',):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	112 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	113 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	114 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	115 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	116 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	117 del x,y,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	118
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	119 def test_minibatch_size(minibatch,minibatch_size,len_ds,nb_field,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	120 ##full minibatch or the last minibatch
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	121 for idx in range(nb_field):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	122 test_minibatch_field_size(minibatch[idx],minibatch_size,len_ds,nb_iter_finished)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	123 del idx
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	124 def test_minibatch_field_size(minibatch_field,minibatch_size,len_ds,nb_iter_finished):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	125 assert len(minibatch_field)==minibatch_size or ((nb_iter_finished*minibatch_size+len(minibatch_field))==len_ds and len(minibatch_field)<minibatch_size)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	126
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	127 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	128 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	129 mi=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	130 m=ds.minibatches(['x','z'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	131 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	132 for minibatch in m:
161 60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	133 assert isinstance(minibatch,DataSetFields)
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	134 assert len(minibatch)==2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	135 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
161 60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	136 if type(ds)==ArrayDataSet:
60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	137 assert (minibatch[0][:,::2]==minibatch[1]).all()
60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	138 else:
162 45427d4d64b3 bugfix in test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 161 diff changeset	139 for j in xrange(len(minibatch[0])):
45427d4d64b3 bugfix in test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 161 diff changeset	140 (minibatch[0][j][::2]==minibatch[1][j]).all()
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	141 mi+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	142 i+=len(minibatch[0])
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	143 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	144 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	145 del minibatch,i,m,mi
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	146
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	147 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	148 mi=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	149 m=ds.minibatches(['x','y'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	150 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	151 for minibatch in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	152 assert len(minibatch)==2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	153 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	154 mi+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	155 for id in range(len(minibatch[0])):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	156 assert (numpy.append(minibatch[0][id],minibatch[1][id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	157 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	158 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	159 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	160 del minibatch,i,id,m,mi
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	161
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	162 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	163 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	164 mi=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	165 m=ds.minibatches(['x','z'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	166 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	167 for x,z in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	168 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	169 test_minibatch_field_size(z,m.minibatch_size,len(ds),mi)
164 3518710e16ec don't assume we have an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 162 diff changeset	170 for id in range(len(x)):
3518710e16ec don't assume we have an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 162 diff changeset	171 assert (x[id][::2]==z[id]).all()
3518710e16ec don't assume we have an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 162 diff changeset	172 i+=1
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	173 mi+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	174 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	175 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	176 del x,z,i,m,mi
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	177 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	178 mi=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	179 m=ds.minibatches(['x','y'], minibatch_size=3)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	180 for x,y in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	181 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	182 test_minibatch_field_size(y,m.minibatch_size,len(ds),mi)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	183 mi+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	184 for id in range(len(x)):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	185 assert (numpy.append(x[id],y[id])==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	186 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	187 assert i==len(ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	188 assert mi==4
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	189 del x,y,i,id,m,mi
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	190
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	191 #not in doc
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	192 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	193 m=ds.minibatches(['x','y'],n_batches=1,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	194 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	195 for x,y in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	196 assert len(x)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	197 assert len(y)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	198 for id in range(3):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	199 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	200 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	201 assert i==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	202 del x,y,i,id,m
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	203
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	204 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	205 m=ds.minibatches(['x','y'],n_batches=2,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	206 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	207 for x,y in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	208 assert len(x)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	209 assert len(y)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	210 for id in range(3):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	211 assert (numpy.append(x[id],y[id])==array[i+4]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	212 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	213 assert i==6
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	214 del x,y,i,id,m
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	215
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	216 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	217 m=ds.minibatches(['x','y'],n_batches=20,minibatch_size=3,offset=4)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	218 assert isinstance(m,DataSet.MinibatchWrapAroundIterator)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	219 for x,y in m:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	220 assert len(x)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	221 assert len(y)==3
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	222 for id in range(3):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	223 assert (numpy.append(x[id],y[id])==array[(i+4)%array.shape[0]]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	224 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	225 assert i==m.n_batches*m.minibatch_size
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	226 del x,y,i,id
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	227
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	228
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	229 def test_ds_iterator(array,iterator1,iterator2,iterator3):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	230 l=len(iterator1)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	231 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	232 for x,y in iterator1:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	233 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	234 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	235 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	236 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	237 assert i==l
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	238 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	239 for y,z in iterator2:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	240 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	241 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	242 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	243 assert i==l
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	244 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	245 for x,y,z in iterator3:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	246 assert (x==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	247 assert y==array[i][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	248 assert (z==array[i][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	249 assert (numpy.append(x,y)==array[i]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	250 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	251 assert i==l
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	252
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	253 def test_getitem(array,ds):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	254 def test_ds(orig,ds,index):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	255 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	256 assert len(ds)==len(index)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	257 for x,z,y in ds('x','z','y'):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	258 assert (orig[index[i]]['x']==array[index[i]][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	259 assert (orig[index[i]]['x']==x).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	260 assert orig[index[i]]['y']==array[index[i]][3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	261 assert orig[index[i]]['y']==y
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	262 assert (orig[index[i]]['z']==array[index[i]][0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	263 assert (orig[index[i]]['z']==z).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	264 i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	265 del i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	266 ds[0]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	267 if len(ds)>2:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	268 ds[:1]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	269 ds[1:1]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	270 ds[1:1:1]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	271 if len(ds)>5:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	272 ds[[1,2,3]]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	273 for x in ds:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	274 pass
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	275
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	276 #ds[:n] returns a dataset with the n first examples.
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	277 ds2=ds[:3]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	278 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	279 test_ds(ds,ds2,index=[0,1,2])
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	280 del ds2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	281
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	282 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s.
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	283 ds2=ds[1:7:2]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	284 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	285 test_ds(ds,ds2,[1,3,5])
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	286 del ds2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	287
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	288 #ds[i]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	289 ds2=ds[5]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	290 assert isinstance(ds2,Example)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	291 assert have_raised("var['ds']["+str(len(ds))+"]",ds=ds) # index not defined
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	292 assert not have_raised("var['ds']["+str(len(ds)-1)+"]",ds=ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	293 del ds2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	294
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	295 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in.
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	296 ds2=ds[[4,7,2,8]]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	297 assert isinstance(ds2,DataSet)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	298 test_ds(ds,ds2,[4,7,2,8])
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	299 del ds2
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	300
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	301 #ds[fieldname]# an iterable over the values of the field fieldname across
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	302 #the ds (the iterable is obtained by default by calling valuesVStack
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	303 #over the values for individual examples).
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	304 assert have_raised("ds['h']") # h is not defined...
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	305 assert have_raised("ds[['x']]") # bad syntax
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	306 assert not have_raised("var['ds']['x']",ds=ds)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	307 isinstance(ds['x'],DataSetFields)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	308 ds2=ds['x']
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	309 assert len(ds['x'])==10
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	310 assert len(ds['y'])==10
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	311 assert len(ds['z'])==10
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	312 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	313 for example in ds['x']:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	314 assert (example==array[i][:3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	315 i+=1
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	316 assert i==len(ds)
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	317 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	318 for example in ds['y']:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	319 assert (example==array[i][3]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	320 i+=1
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	321 assert i==len(ds)
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	322 i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	323 for example in ds['z']:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	324 assert (example==array[i,0:3:2]).all()
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	325 i+=1
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	326 assert i==len(ds)
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	327 del ds2,i
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	328
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	329 #ds.<property># returns the value of a property associated with
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	330 #the name <property>. The following properties should be supported:
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	331 # - 'description': a textual description or name for the ds
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	332 # - 'fieldtypes': a list of types (one per field)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	333
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	334 #* ds1 \| ds2 \| ds3 == ds.hstack([ds1,ds2,ds3])#????
173 1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	335 #assert hstack([ds('x','y'),ds('z')])==ds
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	336 #hstack([ds('z','y'),ds('x')])==ds
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	337 assert have_raised2(hstack,[ds('x'),ds('x')])
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	338 assert have_raised2(hstack,[ds('y','x'),ds('x')])
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	339 assert not have_raised2(hstack,[ds('x'),ds('y')])
1255cd86df36 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 169 diff changeset	340
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	341 # i=0
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	342 # for example in hstack([ds('x'),ds('y'),ds('z')]):
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	343 # example==ds[i]
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	344 # i+=1
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	345 # del i,example
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	346 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3])#????
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	347
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	348 def test_fields_fct(ds):
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	349 #@todo, fill correctly
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	350 assert len(ds.fields())==3
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	351 i=0
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	352 v=0
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	353 for field in ds.fields():
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	354 for field_value in field: # iterate over the values associated to that field for all the ds examples
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	355 v+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	356 i+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	357 assert i==3
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	358 assert v==3*10
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	359 del i,v
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	360
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	361 i=0
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	362 v=0
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	363 for field in ds('x','z').fields():
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	364 i+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	365 for val in field:
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	366 v+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	367 assert i==2
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	368 assert v==2*10
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	369 del i,v
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	370
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	371 i=0
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	372 v=0
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	373 for field in ds.fields('x','y'):
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	374 i+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	375 for val in field:
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	376 v+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	377 assert i==2
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	378 assert v==2*10
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	379 del i,v
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	380
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	381 i=0
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	382 v=0
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	383 for field_examples in ds.fields():
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	384 for example_value in field_examples:
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	385 v+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	386 i+=1
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	387 assert i==3
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	388 assert v==3*10
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	389 del i,v
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	390
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	391 assert ds == ds.fields().examples()
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	392 assert len(ds('x','y').fields()) == 2
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	393 assert len(ds('x','z').fields()) == 2
c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	394 assert len(ds('y').fields()) == 1
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	395
168 c704a66706fe more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 165 diff changeset	396 del field
194 0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	397 def test_all(array,ds):
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	398 assert len(ds)==10
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	399
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	400 test_iterate_over_examples(array, ds)
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	401 test_getitem(array, ds)
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	402 test_ds_iterator(array,ds('x','y'),ds('y','z'),ds('x','y','z'))
0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	403 test_fields_fct(ds)
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	404
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	405 def test_ArrayDataSet():
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	406 #don't test stream
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	407 #tested only with float value
106 cf9bdb1d9656 added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 104 diff changeset	408 #don't always test with y
cf9bdb1d9656 added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 104 diff changeset	409 #don't test missing value
cf9bdb1d9656 added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 104 diff changeset	410 #don't test with tuple
cf9bdb1d9656 added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 104 diff changeset	411 #don't test proterties
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	412 print "test_ArrayDataSet"
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	413 a2 = numpy.random.rand(10,4)
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	414 ds = ArrayDataSet(a2,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested
933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	415 ds = ArrayDataSet(a2,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	416 #assert ds==a? should this work?
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	417
194 0040ba0f0666 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 173 diff changeset	418 test_all(a2,ds)
169 051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True) Frederic Bastien <bastienf@iro.umontreal.ca> parents: 168 diff changeset	419
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	420 del a2, ds
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	421
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	422 def test_LookupList():
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	423 #test only the example in the doc???
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	424 print "test_LookupList"
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	425 example = LookupList(['x','y','z'],[1,2,3])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	426 example['x'] = [1, 2, 3] # set or change a field
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	427 x, y, z = example
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	428 x = example[0]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	429 x = example["x"]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	430 assert example.keys()==['x','y','z']
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	431 assert example.values()==[[1,2,3],2,3]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	432 assert example.items()==[('x',[1,2,3]),('y',2),('z',3)]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	433 example.append_keyval('u',0) # adds item with name 'u' and value 0
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	434 assert len(example)==4 # number of items = 4 here
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	435 example2 = LookupList(['v','w'], ['a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	436 example3 = LookupList(['x','y','z','u','v','w'], [[1, 2, 3],2,3,0,'a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	437 assert example+example2==example3
145 933db7ece663 make some function global to reuse them to test other dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 139 diff changeset	438 assert have_raised("var['x']+var['x']",x=example)
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	439
165 2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	440 del example, example2, example3, x, y ,z
2a12e7437c56 small refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 164 diff changeset	441
158 90104343c665 typo Frederic Bastien <bastienf@iro.umontreal.ca> parents: 157 diff changeset	442 def test_CachedDataSet():
148 a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	443 print "test_CacheDataSet"
169 051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True) Frederic Bastien <bastienf@iro.umontreal.ca> parents: 168 diff changeset	444 a = numpy.random.rand(10,4)
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True) Frederic Bastien <bastienf@iro.umontreal.ca> parents: 168 diff changeset	445 ds1 = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
148 a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	446 ds2 = CachedDataSet(ds1)
a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	447 ds3 = CachedDataSet(ds1,cache_all_upon_construction=True)
a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	448
195 363752267470 typo Frederic Bastien <bastienf@iro.umontreal.ca> parents: 194 diff changeset	449 test_all(a,ds2)
363752267470 typo Frederic Bastien <bastienf@iro.umontreal.ca> parents: 194 diff changeset	450 test_all(a,ds3)
169 051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True) Frederic Bastien <bastienf@iro.umontreal.ca> parents: 168 diff changeset	451
051e07807554 added test for CachedDataSet(ds,cache_all_upon_construction=True) Frederic Bastien <bastienf@iro.umontreal.ca> parents: 168 diff changeset	452 del a,ds1,ds2,ds3
148 a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	453
a5329e719229 added test for CachedDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 145 diff changeset	454
161 60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	455 def test_DataSetFields():
60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	456 print "test_DataSetFields"
60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	457 raise NotImplementedError()
60e00cce3492 bugfix test in case it is not an ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 158 diff changeset	458
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	459 def test_ApplyFunctionDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	460 print "test_ApplyFunctionDataSet"
202 b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	461 a = numpy.random.rand(10,4)
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	462 a2 = a+1
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	463 ds1 = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	464
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	465 ds2 = ApplyFunctionDataSet(ds1,lambda x,y,z: (x+1,y+1,z+1), ['x','y','z'],minibatch_mode=False)
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	466 ds3 = ApplyFunctionDataSet(ds1,lambda x,y,z: (numpy.array(x)+1,numpy.array(y)+1,numpy.array(z)+1), ['x','y','z'],minibatch_mode=True)
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	467
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	468 test_all(a2,ds2)
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	469 test_all(a2,ds3)
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	470
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	471 del a,ds1,ds2,ds3
b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	472
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	473 def test_FieldsSubsetDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	474 print "test_FieldsSubsetDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	475 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	476 def test_MinibatchDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	477 print "test_MinibatchDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	478 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	479 def test_HStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	480 print "test_HStackedDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	481 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	482 def test_VStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	483 print "test_VStackedDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	484 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	485 def test_ArrayFieldsDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	486 print "test_ArrayFieldsDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	487 raise NotImplementedError()
157 e8c8022582f4 only exec the test if we are the main Frederic Bastien <bastienf@iro.umontreal.ca> parents: 149 diff changeset	488 if __name__=='__main__':
e8c8022582f4 only exec the test if we are the main Frederic Bastien <bastienf@iro.umontreal.ca> parents: 149 diff changeset	489 test1()
e8c8022582f4 only exec the test if we are the main Frederic Bastien <bastienf@iro.umontreal.ca> parents: 149 diff changeset	490 test_LookupList()
e8c8022582f4 only exec the test if we are the main Frederic Bastien <bastienf@iro.umontreal.ca> parents: 149 diff changeset	491 test_ArrayDataSet()
158 90104343c665 typo Frederic Bastien <bastienf@iro.umontreal.ca> parents: 157 diff changeset	492 test_CachedDataSet()
202 b9950ae5e54b Added test for ApplyFunctionDataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 195 diff changeset	493 test_ApplyFunctionDataSet()
122 d3c72e412065 removed datat that should not be commited Frederic Bastien <bastienf@iro.umontreal.ca> parents: 116 diff changeset	494 #test pmat.py

Mercurial > pylearn

annotate test_dataset.py @ 212:9b57ea8c767f