pylearn: lookup_list.py annotate

annotate lookup_list.py @ 324:ce79bf5fa463

- the cut and paste between file and dir conditions is always a bad thing - i made one function (hg_version) to basically call and parse hg - i made a function to include the cases of what might be returned by imp.find_modules (_input_id) - the check for a .hg folder was insufficient. Lots of things could go wrong. Instead I use the return code from the Popen process. The return code catches this and any other problem that hg runs into. - its easier to offer more rcs support in future (cvs,svn,git)

author	James Bergstra <bergstrj@iro.umontreal.ca>
date	Thu, 12 Jun 2008 20:54:49 -0400
parents	7c5e5356cb11
children	32c5f87bc54e

rev	line source
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	1
79 427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	2 from copy import deepcopy
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	3
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	4 class LookupList(object):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	5 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	6 A LookupList is a sequence whose elements can be named (and unlike
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	7 a dictionary the order of the elements depends not on their key but
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	8 on the order given by the user through construction) so that
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 142 diff changeset	9 following syntactic constructions work as one would expect::
300 7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	10 >>> example = LookupList(['x','y','z'],[1,2,3])
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	11 >>> example['x'] = [1, 2, 3] # set or change a field
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	12 >>> print example('z','y') # prints [3,2]
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	13 >>> x, y, z = example
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	14 >>> x = example[0]
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	15 >>> x = example["x"]
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	16 >>> print example.keys() # prints ['x','y','z']
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	17 >>> print example.values() # prints [[1,2,3],2,3]
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	18 >>> print example.items() # prints [('x',[1,2,3]),('y',2),('z',3)]
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	19 >>> example.append_keyval('u',0) # adds item with name 'u' and value 0
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	20 >>> print len(example) # number of items = 4 here
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	21 >>> example2 = LookupList(['v', 'w'], ['a','b'])
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	22 >>> print example+example2 # addition is like for lists, a concatenation of the items.
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	23 >>> example + example # throw an error as we can't have duplicate name.
7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	24
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 142 diff changeset	25 @note: The element names should be unique.
300 7c5e5356cb11 doctest syntax for lookup_list James Bergstra <bergstrj@iro.umontreal.ca> parents: 193 diff changeset	26
167 4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 142 diff changeset	27 @todo: Convert this documentation into doctest
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 142 diff changeset	28 format, and actually perform doctest'ing:
4803cb76e26b Updated documentation Joseph Turian <turian@gmail.com> parents: 142 diff changeset	29 U{http://epydoc.sourceforge.net/manual-epytext.html#doctest-blocks}
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	30 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	31 def __init__(self,names=[],values=[]):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	32 assert len(values)==len(names)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	33 self.__dict__['_values']=values
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	34 self.__dict__['_name2index']={}
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	35 self.__dict__['_names']=names
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	36 for i in xrange(len(values)):
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	37 assert names[i] not in self._name2index
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	38 self._name2index[names[i]]=i
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	39
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	40 def keys(self):
17 759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests bergstrj@iro.umontreal.ca parents: 12 diff changeset	41 return self._names
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	42
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	43 def values(self):
17 759d17112b23 more comments, looping ArrayDataSet iterator, bugfixes to lookup_list, more tests bergstrj@iro.umontreal.ca parents: 12 diff changeset	44 return self._values
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	45
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	46 def items(self):
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	47 """
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	48 Return a list of (name,value) pairs of all the items in the look-up list.
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	49 """
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	50 return zip(self._names,self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	51
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	52 def __getitem__(self,key):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	53 """
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	54 The key in example[key] can either be an integer to index the fields
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	55 or the name of the field.
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	56 """
134 3f4e5c9bdc5e Fixes to ApplyFunctionDataSet and other things to make learner and mlp work Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 133 diff changeset	57 if isinstance(key,int) or isinstance(key,slice) or (isinstance(key,list) and all([isinstance(i,int) for i in key])):
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	58 return self._values[key]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	59 else: # if not an int, key must be a name
45 a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: 22 diff changeset	60 # expecting key to be a valid field name
a5c70dc42972 Test functions for dataset.py bengioy@grenat.iro.umontreal.ca parents: 22 diff changeset	61 assert isinstance(key,str)
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	62 return self._values[self._name2index[key]]
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	63
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	64 def __setitem__(self,key,value):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	65 if isinstance(key,int):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	66 self._values[key]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	67 else: # if not an int, key must be a name
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	68 if key in self._name2index:
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	69 self._values[self._name2index[key]]=value
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	70 else:
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	71 self.append_keyval(key,value)
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	72
19 57f4015e2e09 Iterators extend LookupList bergstrj@iro.umontreal.ca parents: 17 diff changeset	73 def append_keyval(self, key, value):
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	74 assert key not in self._name2index
19 57f4015e2e09 Iterators extend LookupList bergstrj@iro.umontreal.ca parents: 17 diff changeset	75 self._name2index[key]=len(self)
57f4015e2e09 Iterators extend LookupList bergstrj@iro.umontreal.ca parents: 17 diff changeset	76 self._values.append(value)
57f4015e2e09 Iterators extend LookupList bergstrj@iro.umontreal.ca parents: 17 diff changeset	77 self._names.append(key)
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	78
137 ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	79 def append_lookuplist(self, *list):
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	80 for l in list:
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	81 for key in l.keys():
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	82 self.append_keyval(key,l[key])
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	83 del l
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	84
12 ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	85 def __len__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	86 return len(self._values)
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	87
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	88 def __repr__(self):
ff4e551490f1 Added LookupList type in lookup_list.py and used it to keep order bengioy@esprit.iro.umontreal.ca parents: diff changeset	89 return "{%s}" % ", ".join([str(k) + "=" + repr(v) for k,v in self.items()])
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	90
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	91 def __add__(self,rhs):
79 427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	92 new_example = deepcopy(self)
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	93 for item in rhs.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	94 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	95 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	96
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	97 def __radd__(self,lhs):
79 427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	98 new_example = deepcopy(lhs)
22 b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	99 for item in self.items():
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	100 new_example.append_keyval(item[0],item[1])
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	101 return new_example
b6b36f65664f Created virtual sub-classes of DataSet: {Finite{Length,Width},Sliceable}DataSet, bengioy@esprit.iro.umontreal.ca parents: 20 diff changeset	102
79 427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	103 def __eq__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	104 return self._values==other._values and self._name2index==other._name2index and self._names==other._names
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	105
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	106 def __ne__(self, other):
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	107 return not self.__eq__(other)
427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	108
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 110 diff changeset	109 def __hash__(self):
79 427e02ef0629 -bugfix. We need to make deep copy otherwise we modify this instance event if we should not do it. Frederic Bastien <bastienf@iro.umontreal.ca> parents: 45 diff changeset	110 raise NotImplementedError()
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 79 diff changeset	111
133 b4657441dd65 Corrected typos Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 110 diff changeset	112 def __call__(self,*names):
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 79 diff changeset	113 """
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 79 diff changeset	114 Return a list of values associated with the given names (which must all be keys of the lookup list).
8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 79 diff changeset	115 """
193 cb6b945acf5a Complete redesign of learner... Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 167 diff changeset	116 if names == self._names:
cb6b945acf5a Complete redesign of learner... Yoshua Bengio <bengioy@iro.umontreal.ca> parents: 167 diff changeset	117 return self._values
110 8fa1ef2411a0 Worked on OneShotTLearner and implementation of LinearRegression bengioy@bengiomac.local parents: 79 diff changeset	118 return [self[name] for name in names]
137 ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	119
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	120
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	121 if __name__ == '__main__':
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	122
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	123 a=LookupList(['a'],[1])
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	124 print a
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	125 b=LookupList(['b'],[2])
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	126 print b
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	127 a.append_lookuplist(b)
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	128 print a
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	129 a.append_lookuplist(b)
ff6b7bfb6cdc added function LookupList.append_lookuplist Frederic Bastien <bastienf@iro.umontreal.ca> parents: 110 diff changeset	130 print a

Mercurial > pylearn

annotate lookup_list.py @ 324:ce79bf5fa463