Mercurial > pylearn

annotate lookup_list.py @ 101:a1740a99b81f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
by default, in a minibatch without any fixed number of batchs, we need to finish at the end of the dataset. Now we return a minibatch at the end event if this minibacht size != the gived minibatch_size.
author Frederic Bastien <bastienf@iro.umontreal.ca>
date Tue, 06 May 2008 16:01:53 -0400
parents 427e02ef0629
children 8fa1ef2411a0
rev   line source
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
1
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
2 from copy import deepcopy
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
3
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
4 class LookupList(object):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
5 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
6 A LookupList is a sequence whose elements can be named (and unlike
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
7 a dictionary the order of the elements depends not on their key but
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
8 on the order given by the user through construction) so that
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
9 following syntactic constructions work as one would expect:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
10 example = LookupList(['x','y','z'],[1,2,3])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
11 example['x'] = [1, 2, 3] # set or change a field
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
12 x, y, z = example
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
13 x = example[0]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
14 x = example["x"]
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
15 print example.keys() # prints ['x','y','z']
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
16 print example.values() # prints [[1,2,3],2,3]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
17 print example.items() # prints [('x',[1,2,3]),('y',2),('z',3)]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
18 example.append_keyval('u',0) # adds item with name 'u' and value 0
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
19 print len(example) # number of items = 4 here
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
20 example2 = LookupList(['v', 'w'], ['a','b'])
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
21 print example+example2 # addition is like for lists, a concatenation of the items.
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
22 example + example # throw an error as we can't have duplicate name.
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
23 Note that the element names should be unique.
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
24 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
25 def __init__(self,names=[],values=[]):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
26 assert len(values)==len(names)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
27 self.__dict__['_values']=values
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
28 self.__dict__['_name2index']={}
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
29 self.__dict__['_names']=names
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
30 for i in xrange(len(values)):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
31 assert names[i] not in self._name2index
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
32 self._name2index[names[i]]=i
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
33
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
34 def keys(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
35 return self._names
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
36
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
37 def values(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
38 return self._values
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
39
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
40 def items(self):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
41 """
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
42 Return a list of (name,value) pairs of all the items in the look-up list.
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
43 """
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
44 return zip(self._names,self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
45
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
46 def __getitem__(self,key):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
47 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
48 The key in example[key] can either be an integer to index the fields
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
49 or the name of the field.
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
50 """
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
51 if isinstance(key,int) or isinstance(key,slice) or isinstance(key,list):
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
52 return self._values[key]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
53 else: # if not an int, key must be a name
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
54 # expecting key to be a valid field name
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
55 assert isinstance(key,str)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
56 return self._values[self._name2index[key]]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
57
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
58 def __setitem__(self,key,value):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
59 if isinstance(key,int):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
60 self._values[key]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
61 else: # if not an int, key must be a name
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
62 if key in self._name2index:
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
63 self._values[self._name2index[key]]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
64 else:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
65 self.append_keyval(key,value)
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
66
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
67 def append_keyval(self, key, value):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
68 assert key not in self._name2index
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
69 self._name2index[key]=len(self)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
70 self._values.append(value)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
71 self._names.append(key)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
72
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
73 def __len__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
74 return len(self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
75
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
76 def __repr__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
77 return "{%s}" % ", ".join([str(k) + "=" + repr(v) for k,v in self.items()])
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
78
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
79 def __add__(self,rhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
80 new_example = deepcopy(self)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
81 for item in rhs.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
82 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
83 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
84
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
85 def __radd__(self,lhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
86 new_example = deepcopy(lhs)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
87 for item in self.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
88 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
89 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
90
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
91
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
92 def __eq__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
93 return self._values==other._values and self._name2index==other._name2index and self._names==other._names
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
94
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
95 def __ne__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
96 return not self.__eq__(other)
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
97
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
98 def __hash__():
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
99 raise NotImplementedError()