Mercurial > pylearn

annotate lookup_list.py @ 146:8173e196e291

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Trying to make CacheDataSet work
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Mon, 12 May 2008 15:50:28 -0400
parents ad144fa72bf5
children 4803cb76e26b
rev   line source
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
1
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
2 from copy import deepcopy
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
3
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
4 class LookupList(object):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
5 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
6 A LookupList is a sequence whose elements can be named (and unlike
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
7 a dictionary the order of the elements depends not on their key but
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
8 on the order given by the user through construction) so that
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
9 following syntactic constructions work as one would expect:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
10 example = LookupList(['x','y','z'],[1,2,3])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
11 example['x'] = [1, 2, 3] # set or change a field
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
12 print example('z','y') # prints [3,2]
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
13 x, y, z = example
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
14 x = example[0]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
15 x = example["x"]
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
16 print example.keys() # prints ['x','y','z']
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
17 print example.values() # prints [[1,2,3],2,3]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
18 print example.items() # prints [('x',[1,2,3]),('y',2),('z',3)]
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
19 example.append_keyval('u',0) # adds item with name 'u' and value 0
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
20 print len(example) # number of items = 4 here
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
21 example2 = LookupList(['v', 'w'], ['a','b'])
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
22 print example+example2 # addition is like for lists, a concatenation of the items.
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
23 example + example # throw an error as we can't have duplicate name.
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
24 Note that the element names should be unique.
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
25 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
26 def __init__(self,names=[],values=[]):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
27 assert len(values)==len(names)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
28 self.__dict__['_values']=values
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
29 self.__dict__['_name2index']={}
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
30 self.__dict__['_names']=names
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
31 for i in xrange(len(values)):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
32 assert names[i] not in self._name2index
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
33 self._name2index[names[i]]=i
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
34
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
35 def keys(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
36 return self._names
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
37
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
38 def values(self):
17
759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests
bergstrj@iro.umontreal.ca
parents: 12
diff changeset
39 return self._values
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
40
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
41 def items(self):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
42 """
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
43 Return a list of (name,value) pairs of all the items in the look-up list.
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
44 """
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
45 return zip(self._names,self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
46
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
47 def __getitem__(self,key):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
48 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
49 The key in example[key] can either be an integer to index the fields
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
50 or the name of the field.
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
51 """
134
3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 133
diff changeset
52 if isinstance(key,int) or isinstance(key,slice) or (isinstance(key,list) and all([isinstance(i,int) for i in key])):
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
53 return self._values[key]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
54 else: # if not an int, key must be a name
45
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
55 # expecting key to be a valid field name
a5c70dc42972 Test functions for dataset.py
bengioy@grenat.iro.umontreal.ca
parents: 22
diff changeset
56 assert isinstance(key,str)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
57 return self._values[self._name2index[key]]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
58
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
59 def __setitem__(self,key,value):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
60 if isinstance(key,int):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
61 self._values[key]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
62 else: # if not an int, key must be a name
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
63 if key in self._name2index:
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
64 self._values[self._name2index[key]]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
65 else:
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
66 self.append_keyval(key,value)
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
67
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
68 def append_keyval(self, key, value):
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
69 assert key not in self._name2index
19
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
70 self._name2index[key]=len(self)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
71 self._values.append(value)
57f4015e2e09 Iterators extend LookupList
bergstrj@iro.umontreal.ca
parents: 17
diff changeset
72 self._names.append(key)
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
73
137
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
74 def append_lookuplist(self, *list):
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
75 for l in list:
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
76 for key in l.keys():
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
77 self.append_keyval(key,l[key])
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
78 del l
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
79
12
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
80 def __len__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
81 return len(self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
82
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
83 def __repr__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order
bengioy@esprit.iro.umontreal.ca
parents:
diff changeset
84 return "{%s}" % ", ".join([str(k) + "=" + repr(v) for k,v in self.items()])
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
85
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
86 def __add__(self,rhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
87 new_example = deepcopy(self)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
88 for item in rhs.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
89 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
90 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
91
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
92 def __radd__(self,lhs):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
93 new_example = deepcopy(lhs)
22
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
94 for item in self.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
95 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
96 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet,
bengioy@esprit.iro.umontreal.ca
parents: 20
diff changeset
97
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
98 def __eq__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
99 return self._values==other._values and self._name2index==other._name2index and self._names==other._names
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
100
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
101 def __ne__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
102 return not self.__eq__(other)
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
103
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 110
diff changeset
104 def __hash__(self):
79
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it.
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 45
diff changeset
105 raise NotImplementedError()
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
106
133
b4657441dd65 Corrected typos
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 110
diff changeset
107 def __call__(self,*names):
110
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
108 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
109 Return a list of values associated with the given names (which must all be keys of the lookup list).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
110 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression
bengioy@bengiomac.local
parents: 79
diff changeset
111 return [self[name] for name in names]
137
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
112
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
113
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
114 if __name__ == '__main__':
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
115
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
116 a=LookupList(['a'],[1])
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
117 print a
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
118 b=LookupList(['b'],[2])
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
119 print b
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
120 a.append_lookuplist(b)
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
121 print a
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
122 a.append_lookuplist(b)
ff6b7bfb6cdc added function LookupList.append_lookuplist
Frederic Bastien <bastienf@iro.umontreal.ca>
parents: 110
diff changeset
123 print a