Mercurial > pylearn

annotate test_speed.py @ 431:0f8c81b0776d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Adding file make_test_datasets to host simple data-generating processes to create artificial datasets meant to test various learning algorithms.
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Tue, 29 Jul 2008 10:19:25 -0400
parents 6255359318bf
children
rev   line source
277
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
1 import numpy
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
2 from dataset import *
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
3 from misc import *
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
4 def test_speed(array, ds):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
5 print "test_speed", ds.__class__
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
6
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
7 mat = numpy.random.rand(400,100)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
8
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
9 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
10 def f_array_full(a):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
11 a+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
12 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
13 def f_array_index(a):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
14 for id in range(a.shape[0]):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
15 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
16 a[id]+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
17 # a[id]*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
18 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
19 def f_array_iter(a):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
20 for r in a:
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
21 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
22 r+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
23 # r*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
24 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
25 def f_ds_index(ds):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
26 for id in range(len(ds)):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
27 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
28 ds[id][0]+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
29 # ds[id][0]*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
30 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
31 def f_ds_iter(ds):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
32 for ex in ds:
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
33 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
34 ex[0]+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
35 # a[0]*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
36 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
37 def f_ds_mb1(ds,mb_size):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
38 for exs in ds.minibatches(minibatch_size = mb_size):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
39 for ex in exs:
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
40 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
41 ex[0]+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
42 # ex[0]*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
43 @print_timing
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
44 def f_ds_mb2(ds,mb_size):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
45 for exs in ds.minibatches(minibatch_size = mb_size):
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
46 # pass
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
47 exs[0]+1
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
48 # ex[0]*mat
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
49
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
50 f_array_full(array)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
51 f_array_index(array)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
52 f_array_iter(array)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
53
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
54 f_ds_index(ds)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
55 f_ds_iter(ds)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
56
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
57 f_ds_mb1(ds,10)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
58 f_ds_mb1(ds,100)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
59 f_ds_mb1(ds,1000)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
60 f_ds_mb1(ds,10000)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
61 f_ds_mb2(ds,10)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
62 f_ds_mb2(ds,100)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
63 f_ds_mb2(ds,1000)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
64 f_ds_mb2(ds,10000)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
65
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
66 if __name__=='__main__':
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
67 a2 = numpy.random.rand(100000,400)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
68 ds1 = ArrayDataSet(a2,{'all':slice(0,a2.shape[1],1)})
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
69 test_speed(a2,ds1)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
70 a1 = numpy.random.rand(100000,40)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
71 ds4 = ArrayDataSet(a1,LookupList(["f"+str(x)for x in range(a1.shape[1])],
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
72 range(a1.shape[1])))
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
73 test_speed(a2,ds4)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
74 ds2=CachedDataSet(ds1,cache_all_upon_construction=False)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
75 test_speed(a2,ds2)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
76 ds3=CachedDataSet(ds1,cache_all_upon_construction=True)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
77 test_speed(a2,ds3)
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
78 del a2,ds1,ds2,ds3
6255359318bf moved test_speed() in its own file
Frederic Bastien <bastienf@iro.umontreal.ca>
parents:
diff changeset
79