pylearn: test_dataset.py annotate

annotate test_dataset.py @ 102:4537ac630348

modifed test to accomodate the last change in dataset.py. i.e. minibatch without a fixed number of batch return an incomplete minibatch at the end to stop at the end of the dataset.

author	Frederic Bastien <bastienf@iro.umontreal.ca>
date	Tue, 06 May 2008 16:03:17 -0400
parents	574f4db76022
children	a90d85fef3d4

rev	line source
51 59757365a057 the script can be autorun Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	1 #!/bin/env python
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	2 from dataset import *
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	3 from math import *
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	4 import numpy
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	5
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	6 def have_raised(to_eval):
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	7 have_thrown = False
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	8 try:
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	9 eval(to_eval)
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	10 except :
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	11 have_thrown = True
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	12 return have_thrown
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	13
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	14 def test1():
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	15 print "test1"
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	16 global a,ds
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	17 a = numpy.random.rand(10,4)
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	18 print a
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	19 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	20 print "len(ds)=",len(ds)
54 70147d00615a added assert Frederic Bastien <bastienf@iro.umontreal.ca> parents: 52 diff changeset	21 assert(len(ds)==10)
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	22 print "example 0 = ",ds[0]
54 70147d00615a added assert Frederic Bastien <bastienf@iro.umontreal.ca> parents: 52 diff changeset	23 # assert
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	24 print "x=",ds["x"]
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	25 print "x\|y"
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	26 for x,y in ds("x","y"):
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	27 print x,y
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	28 minibatch_iterator = ds.minibatches(fieldnames=['z','y'],n_batches=1,minibatch_size=3,offset=4)
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	29 minibatch = minibatch_iterator.__iter__().next()
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	30 print "minibatch=",minibatch
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	31 for var in minibatch:
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	32 print "var=",var
48 b6730f9a336d Fixing MinibatchDataSet getitem bengioy@grenat.iro.umontreal.ca parents: 45 diff changeset	33 print "take a slice and look at field y",ds[1:6:2]["y"]
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: diff changeset	34
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	35 def test_ArrayDataSet():
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	36 #don't test stream
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	37 #tested only with float value
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	38 #test with y too
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	39 #test missing value
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	40
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	41 def test_iterate_over_examples(array,ds):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	42 #not in doc!!!
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	43 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	44 for example in range(len(ds)):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	45 assert (ds[example]['x']==a[example][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	46 assert ds[example]['y']==a[example][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	47 assert (ds[example]['z']==a[example][[0,2]]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	48 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	49 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	50 del example,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	51
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	52 # - for example in dataset:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	53 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	54 for example in ds:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	55 assert len(example)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	56 assert (example['x']==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	57 assert example['y']==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	58 assert (example['z']==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	59 assert (numpy.append(example['x'],example['y'])==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	60 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	61 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	62 del example,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	63
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	64 # - for val1,val2,... in dataset:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	65 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	66 for x,y,z in ds:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	67 assert (x==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	68 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	69 assert (z==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	70 assert (numpy.append(x,y)==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	71 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	72 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	73 del x,y,z,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	74
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	75 # - for example in dataset(field1, field2,field3, ...):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	76 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	77 for example in ds('x','y','z'):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	78 assert len(example)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	79 assert (example['x']==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	80 assert example['y']==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	81 assert (example['z']==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	82 assert (numpy.append(example['x'],example['y'])==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	83 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	84 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	85 del example,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	86 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	87 for example in ds('y','x'):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	88 assert len(example)==2
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	89 assert (example['x']==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	90 assert example['y']==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	91 assert (numpy.append(example['x'],example['y'])==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	92 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	93 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	94 del example,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	95
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	96 # - for val1,val2,val3 in dataset(field1, field2,field3):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	97 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	98 for x,y,z in ds('x','y','z'):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	99 assert (x==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	100 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	101 assert (z==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	102 assert (numpy.append(x,y)==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	103 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	104 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	105 del x,y,z,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	106 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	107 for y,x in ds('y','x',):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	108 assert (x==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	109 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	110 assert (numpy.append(x,y)==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	111 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	112 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	113 del x,y,i
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	114
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	115 def test_minibatch_size(minibatch,minibatch_size,len_ds,nb_field,nb_iter_finished):
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	116 ##full minibatch or the last minibatch
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	117 for idx in range(nb_field):
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	118 test_minibatch_field_size(minibatch[idx],minibatch_size,len_ds,nb_iter_finished)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	119 del idx
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	120 def test_minibatch_field_size(minibatch_field,minibatch_size,len_ds,nb_iter_finished):
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	121 assert len(minibatch_field)==minibatch_size or ((nb_iter_finished*minibatch_size+len(minibatch_field))==len_ds and len(minibatch_field)<minibatch_size)
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	122
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	123 # - for minibatch in dataset.minibatches([field1, field2, ...],minibatch_size=N):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	124 i=0
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	125 mi=0
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	126 m=ds.minibatches(['x','z'], minibatch_size=3)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	127 for minibatch in m:
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	128 assert len(minibatch)==2
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	129 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	130 assert (minibatch[0][:,0:3:2]==minibatch[1]).all()
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	131 mi+=1
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	132 i+=len(minibatch[0])
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	133 assert i==len(ds)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	134 assert mi==4
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	135 del minibatch,i,m,mi
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	136
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	137 i=0
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	138 mi=0
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	139 m=ds.minibatches(['x','y'], minibatch_size=3)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	140 for minibatch in m:
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	141 assert len(minibatch)==2
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	142 test_minibatch_size(minibatch,m.minibatch_size,len(ds),2,mi)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	143 mi+=1
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	144 for id in range(len(minibatch[0])):
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	145 assert (numpy.append(minibatch[0][id],minibatch[1][id])==a[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	146 i+=1
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	147 assert i==len(ds)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	148 assert mi==4
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	149 del minibatch,i,id,m,mi
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	150
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	151 # - for mini1,mini2,mini3 in dataset.minibatches([field1, field2, field3], minibatch_size=N):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	152 i=0
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	153 mi=0
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	154 m=ds.minibatches(['x','z'], minibatch_size=3)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	155 for x,z in m:
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	156 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	157 test_minibatch_field_size(z,m.minibatch_size,len(ds),mi)
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	158 assert (x[:,0:3:2]==z).all()
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	159 i+=len(x)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	160 mi+=1
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	161 assert i==len(ds)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	162 assert mi==4
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	163 del x,z,i,m,mi
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	164 i=0
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	165 mi=0
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	166 m=ds.minibatches(['x','y'], minibatch_size=3)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	167 for x,y in m:
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	168 test_minibatch_field_size(x,m.minibatch_size,len(ds),mi)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	169 test_minibatch_field_size(y,m.minibatch_size,len(ds),mi)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	170 mi+=1
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	171 for id in range(len(x)):
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	172 assert (numpy.append(x[id],y[id])==a[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	173 i+=1
102 4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	174 assert i==len(ds)
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	175 assert mi==4
4537ac630348 modifed test to accomodate the last change in dataset.py. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 100 diff changeset	176 del x,y,i,id,m,mi
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	177
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	178 #not in doc
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	179 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	180 for x,y in ds.minibatches(['x','y'],n_batches=1,minibatch_size=3,offset=4):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	181 assert len(x)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	182 assert len(y)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	183 for id in range(3):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	184 assert (numpy.append(x[id],y[id])==a[i+4]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	185 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	186 assert i==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	187 del x,y,i,id
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	188
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	189 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	190 for x,y in ds.minibatches(['x','y'],n_batches=2,minibatch_size=3,offset=4):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	191 assert len(x)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	192 assert len(y)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	193 for id in range(3):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	194 assert (numpy.append(x[id],y[id])==a[i+4]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	195 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	196 assert i==6
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	197 del x,y,i,id
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	198
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	199 i=0
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	200 m=ds.minibatches(['x','y'],n_batches=10,minibatch_size=3,offset=4) # bugged???
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	201 for x,y in m:
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	202 assert len(x)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	203 assert len(y)==3
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	204 for id in range(3):
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	205 assert (numpy.append(x[id],y[id])==a[(i+4)%a.shape[0]]).all()
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	206 i+=1
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	207 assert i==m.n_batches*m.minibatch_size
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	208 del x,y,i,id
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	209
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	210
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	211 def test_ds_iterator(array,iterator1,iterator2,iterator3):
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	212 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	213 for x,y in iterator1:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	214 assert (x==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	215 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	216 assert (numpy.append(x,y)==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	217 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	218 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	219 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	220 for y,z in iterator2:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	221 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	222 assert (z==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	223 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	224 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	225 i=0
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	226 for x,y,z in iterator3:
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	227 assert (x==array[i][:3]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	228 assert y==array[i][3]
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	229 assert (z==array[i][0:3:2]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	230 assert (numpy.append(x,y)==array[i]).all()
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	231 i+=1
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	232 assert i==len(ds)
352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	233
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	234 def test_getitem(array,ds):
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	235
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	236 def test_ds(orig,ds,index):
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	237 i=0
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	238 assert len(ds)==len(index)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	239 for x,z,y in ds('x','z','y'):
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	240 assert (orig[index[i]]['x']==array[index[i]][:3]).all()
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	241 assert (orig[index[i]]['x']==x).all()
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	242 assert orig[index[i]]['y']==array[index[i]][3]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	243 assert orig[index[i]]['y']==y
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	244 assert (orig[index[i]]['z']==array[index[i]][0:3:2]).all()
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	245 assert (orig[index[i]]['z']==z).all()
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	246 i+=1
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	247 del i
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	248 ds[0]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	249 if len(ds)>2:
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	250 ds[:1]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	251 ds[1:1]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	252 ds[1:1:1]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	253 if len(ds)>5:
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	254 ds[[1,2,3]]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	255 for x in ds:
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	256 pass
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	257
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	258 assert have_raised("ds['h']") # h is not defined...
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	259 assert have_raised("ds["+str(len(ds))+"]") # h is not defined...
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	260 assert have_raised("ds["+str(len(ds))+"]") # h is not defined...
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	261 assert have_raised("ds[['h']]") # h is not defined...
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	262
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	263 #ds[:n] returns a dataset with the n first examples.
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	264 ds2=ds[:3]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	265 assert isinstance(ds2,DataSet)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	266 test_ds(ds,ds2,index=[0,1,2])
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	267
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	268 #ds[i1:i2:s]# returns a ds with the examples i1,i1+s,...i2-s.
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	269 ds2=ds[1:7:2]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	270 assert isinstance(ds2,DataSet)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	271 test_ds(ds,ds2,[1,3,5])
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	272
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	273 #ds[i]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	274 ds2=ds[5]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	275 assert isinstance(ds2,Example)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	276
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	277 #ds[[i1,i2,...in]]# returns a ds with examples i1,i2,...in.
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	278 ds2=ds[[4,7,2,8]]
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	279 assert isinstance(ds2,DataSet)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	280 test_ds(ds,ds2,[4,7,2,8])
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	281 #ds[i1,i2,...]# should we accept????no syntax of numpy.array
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	282
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	283 #ds[fieldname]# an iterable over the values of the field fieldname across
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	284 #the ds (the iterable is obtained by default by calling valuesVStack
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	285 #over the values for individual examples).
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	286
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	287 #ds.<property># returns the value of a property associated with
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	288 #the name <property>. The following properties should be supported:
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	289 # - 'description': a textual description or name for the ds
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	290 # - 'fieldtypes': a list of types (one per field)
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	291 #* ds1 \| ds2 \| ds3 == ds.hstack([ds1,ds2,ds3])#????
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	292 #* ds1 & ds2 & ds3 == ds.vstack([ds1,ds2,ds3])#????
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	293
574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	294
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	295 print "test_ArrayDataSet"
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	296 a = numpy.random.rand(10,4)
86 fdf72ea4f2bc added function test_ds in test_ArrayDataSet who test a sub dataset Frederic Bastien <bastienf@iro.umontreal.ca> parents: 84 diff changeset	297 ds = ArrayDataSet(a,{'x':slice(3),'y':3,'z':[0,2]})###???tuple not tested
91 eee739fefdff corrected test from discution about the syntax with Yoshua Frederic Bastien <bastienf@iro.umontreal.ca> parents: 89 diff changeset	298 ds = ArrayDataSet(a,LookupList(['x','y','z'],[slice(3),3,[0,2]]))###???tuple not tested
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	299 assert len(ds)==10
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	300 #assert ds==a? should this work?
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	301
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	302 test_iterate_over_examples(a, ds)
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	303 test_getitem(a, ds)
91 eee739fefdff corrected test from discution about the syntax with Yoshua Frederic Bastien <bastienf@iro.umontreal.ca> parents: 89 diff changeset	304
eee739fefdff corrected test from discution about the syntax with Yoshua Frederic Bastien <bastienf@iro.umontreal.ca> parents: 89 diff changeset	305 # - for val1,val2,val3 in dataset(field1, field2,field3):
96 352910e0dbf5 added test and some restructuring for futur use Frederic Bastien <bastienf@iro.umontreal.ca> parents: 94 diff changeset	306 test_ds_iterator(a,ds('x','y'),ds('y','z'),ds('x','y','z'))
89 05dc4804357b more test and refactoring Frederic Bastien <bastienf@iro.umontreal.ca> parents: 87 diff changeset	307
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	308
63 14589f02a372 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 60 diff changeset	309 assert len(ds.fields())==3
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	310 for field in ds.fields():
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	311 for field_value in field: # iterate over the values associated to that field for all the ds examples
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	312 pass
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	313 for field in ds('x','z').fields():
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	314 pass
60 9165d86855ab bugfix Frederic Bastien <bastienf@iro.umontreal.ca> parents: 58 diff changeset	315 for field in ds.fields('x','y'):
58 17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	316 pass
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	317 for field_examples in ds.fields():
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	318 for example_value in field_examples:
17729d7104fa added function test_ArrayDataSet Frederic Bastien <bastienf@iro.umontreal.ca> parents: 54 diff changeset	319 pass
63 14589f02a372 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 60 diff changeset	320
14589f02a372 more test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 60 diff changeset	321 assert ds == ds.fields().examples()
100 574f4db76022 restructuring and added test Frederic Bastien <bastienf@iro.umontreal.ca> parents: 96 diff changeset	322 # for ((x,y),a_v) in (ds('x','y'),a): #???don't work # haven't found a variant that work.# will not work
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	323 # assert numpy.append(x,y)==z
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	324
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	325 def test_LookupList():
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	326 #test only the example in the doc???
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	327 print "test_LookupList"
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	328 example = LookupList(['x','y','z'],[1,2,3])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	329 example['x'] = [1, 2, 3] # set or change a field
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	330 x, y, z = example
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	331 x = example[0]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	332 x = example["x"]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	333 assert example.keys()==['x','y','z']
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	334 assert example.values()==[[1,2,3],2,3]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	335 assert example.items()==[('x',[1,2,3]),('y',2),('z',3)]
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	336 example.append_keyval('u',0) # adds item with name 'u' and value 0
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	337 assert len(example)==4 # number of items = 4 here
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	338 example2 = LookupList(['v','w'], ['a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	339 example3 = LookupList(['x','y','z','u','v','w'], [[1, 2, 3],2,3,0,'a','b'])
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	340 assert example+example2==example3
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	341 assert have_raised("example+example")
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	342
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	343 def test_ApplyFunctionDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	344 print "test_ApplyFunctionDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	345 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	346 def test_CacheDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	347 print "test_CacheDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	348 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	349 def test_FieldsSubsetDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	350 print "test_FieldsSubsetDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	351 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	352 def test_DataSetFields():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	353 print "test_DataSetFields"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	354 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	355 def test_MinibatchDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	356 print "test_MinibatchDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	357 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	358 def test_HStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	359 print "test_HStackedDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	360 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	361 def test_VStackedDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	362 print "test_VStackedDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	363 raise NotImplementedError()
84 aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	364 def test_ArrayFieldsDataSet():
aa9e786ee849 added function have_raised that evaluate the string in parameter and return true if the function have raised an exception Frederic Bastien <bastienf@iro.umontreal.ca> parents: 82 diff changeset	365 print "test_ArrayFieldsDataSet"
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	366 raise NotImplementedError()
4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	367
74 b4159cbdc06b Fixed errors raised by test_dataset Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 66 diff changeset	368 test1()
81 4b0859606d05 Added test for ArrayDataSet and LookUpList Frederic Bastien <bastienf@iro.umontreal.ca> parents: 66 diff changeset	369 test_LookupList()
65 d48eba49a2f4 fixed the infinite loop Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 64 diff changeset	370 test_ArrayDataSet()
64 863da25a60f1 trying to fix infinite loop Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 63 diff changeset	371

Mercurial > pylearn

annotate test_dataset.py @ 102:4537ac630348