Mercurial > pylearn

annotate lookup_list.py @ 99:a8da709eb6a9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
in ArrayDataSet.__init__ if a columns is an index, we change it to be a list that containt only this index. This way, we remove the special case where the columns is an index for all subsequent call. This was possing trouble with numpy.vstack() called by MinibatchWrapAroundIterator.next
author Frederic Bastien <bastienf@iro.umontreal.ca>
date Tue, 06 May 2008 13:57:36 -0400
parents 427e02ef0629
children 8fa1ef2411a0
rev   line source
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
1
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
2 from copy import deepcopy
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
3
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
4 class LookupList(object):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
5 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
6 A LookupList is a sequence whose elements can be named (and unlike
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
7 a dictionary the order of the elements depends not on their key but
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
8 on the order given by the user through construction) so that
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
9 following syntactic constructions work as one would expect:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
10 example = LookupList(['x','y','z'],[1,2,3])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
11 example['x'] = [1, 2, 3] # set or change a field
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
12 x, y, z = example
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
13 x = example[0]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
14 x = example["x"]
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
15 print example.keys() # prints ['x','y','z']
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
16 print example.values() # prints [[1,2,3],2,3]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
17 print example.items() # prints [('x',[1,2,3]),('y',2),('z',3)]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
18 example.append_keyval('u',0) # adds item with name 'u' and value 0
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
19 print len(example) # number of items = 4 here
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
20 example2 = LookupList(['v', 'w'], ['a','b'])
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
21 print example+example2 # addition is like for lists, a concatenation of the items.
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
22 example + example # throw an error as we can't have duplicate name.
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
23 Note that the element names should be unique.
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
24 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
25 def __init__(self,names=[],values=[]):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
26 assert len(values)==len(names)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
27 self.__dict__['_values']=values
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
28 self.__dict__['_name2index']={}
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
29 self.__dict__['_names']=names
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
30 for i in xrange(len(values)):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
31 assert names[i] not in self._name2index
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
32 self._name2index[names[i]]=i
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
33
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
34 def keys(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
35 return self._names
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
36
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
37 def values(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
38 return self._values
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
39
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
40 def items(self):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
41 """
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
42 Return a list of (name,value) pairs of all the items in the look-up list.
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
43 """
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
44 return zip(self._names,self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
45
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
46 def __getitem__(self,key):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
47 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
48 The key in example[key] can either be an integer to index the fields
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
49 or the name of the field.
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
50 """
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
51 if isinstance(key,int) or isinstance(key,slice) or isinstance(key,list):
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
52 return self._values[key]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
53 else: # if not an int, key must be a name
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
54 # expecting key to be a valid field name
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
55 assert isinstance(key,str)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
56 return self._values[self._name2index[key]]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
57
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
58 def __setitem__(self,key,value):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
59 if isinstance(key,int):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
60 self._values[key]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
61 else: # if not an int, key must be a name
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
62 if key in self._name2index:
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
63 self._values[self._name2index[key]]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
64 else:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
65 self.append_keyval(key,value)
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
66
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
67 def append_keyval(self, key, value):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
68 assert key not in self._name2index
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
69 self._name2index[key]=len(self)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
70 self._values.append(value)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
71 self._names.append(key)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
72
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
73 def __len__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
74 return len(self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
75
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
76 def __repr__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
77 return "{%s}" % ", ".join([str(k) + "=" + repr(v) for k,v in self.items()])
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
78
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
79 def __add__(self,rhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
80 new_example = deepcopy(self)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
81 for item in rhs.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
82 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
83 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
84
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
85 def __radd__(self,lhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
86 new_example = deepcopy(lhs)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
87 for item in self.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
88 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
89 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
90
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
91
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
92 def __eq__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
93 return self._values==other._values and self._name2index==other._name2index and self._names==other._names
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
94
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
95 def __ne__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
96 return not self.__eq__(other)
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
97
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
98 def __hash__():
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
99 raise NotImplementedError()