pylearn: pylearn/sandbox/scan_inputs

annotate pylearn/sandbox/scan_inputs_groups.py @ 701:113946723973

fixed bug of scan_inputs_groups

author	Xavier Glorot <glorotxa@iro.umontreal.ca>
date	Tue, 19 May 2009 19:00:34 -0400
parents	33e13d8ca7d3
children	0eae6d5315b5

rev	line source
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	1 import numpy
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	2 import theano
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	3 from theano import tensor as T
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	4 from theano.gof import Op
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	5 from theano.gof import Apply
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	6 from theano import scalar as scal
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	7
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	8 # These Ops allows us to deal with static groups of possibly missing inputs efficiently in the dense DAA framework
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	9 # (for exemple with multimodal data with sometimes entire modality missing).
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	10 # The inputs will be represented with an index list and a theano.generic variable (which will be a list of matrices
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	11 # (numpy array), each element will correspond to an available modality and the index list will indicates the weights
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	12 # associated to it).
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	13 # Exemple of index list: [1, 0, -3]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	14 # *the 1 says that the first element of the input list will refer to the first element of the weights_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	15 # (auxiliary target as input)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	16 # if inputslist[i]>0 it refers to Weightslist[indexlist[i]-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	17 # *the 0 means that the second element of the input list will not be encoded neither decoded (it is remplaced by zeros)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	18 # this is not efficient, so in this case it is better to give: [1,-3] and [inputslist[0],inputslist[2]]
700 33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	19 # but it allows us to deal with empty lists: give indexlist = numpy.asarray([.0])
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	20 # and inputlist=numpy.zeros((batchsize,1))
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	21 # *when an index is negative it means that the input will not be used for encoding but we will still reconstruct it
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	22 # (auxiliary target as output)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	23 # if inputslist[i]<0 it refers to Weightslist[-indexlist[i]-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	24 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	25 # An entire batch should have the same available inputs configuration.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	26 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	27 # Dense DAA Exemple:----------------------------------------------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	28 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	29 #from theano.tensor.nnet import sigmoid
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	30 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	31 #nb_modality = 4
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	32 #wenc = [T.dmatrix('wenc%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	33 #wdec = [T.dmatrix('wdec%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	34 #benc = T.dvector('benc')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	35 #bdec = [T.dvector('bdec%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	36 #vectin = T.ivector('vectin')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	37 #inputpart = theano.generic('inputpart')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	38 #noise_bit = T.dscalar('noise_bit')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	39 #noise_group = T.dscalar('noise_group')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	40 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	41 #[vectin2,inputpart2] = scannoise(vectin,inputpart,noise_bit,noise_group)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	42 #hid = scandotenc(vectin2, inputpart2, wenc)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	43 #acthid = sigmoid(hid + benc)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	44 #dec = sigmoid(scanbiasdec(vectin2,inputpart2,bdec) + scandotdec(vectin2, inputpart2,acthid,wdec))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	45 #cost = T.sum(T.sum(T.sqr( scaninput(vectin,inputpart) - rec ),1),0)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	46
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	47 # Checking inputs in make_node methods----------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	48 def Checkidx_list(idx_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	49 idx_list = T.as_tensor_variable(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	50 nidx = idx_list.type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	51 if nidx != 1: raise TypeError('not vector', idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	52 return idx_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	53
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	54 def Checkhidd(hidd):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	55 hidd = T.as_tensor_variable(hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	56 nhidd = hidd.type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	57 if nhidd not in (1,2): raise TypeError('not matrix or vector', hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	58 return hidd
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	59
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	60 def Checkweights_list(weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	61 weights_list = map(T.as_tensor_variable, weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	62 for i in range(len(weights_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	63 nweights = weights_list[i].type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	64 if nweights not in (1,2): raise TypeError('not matrix or vector', weights_list[i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	65 return weights_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	66
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	67 def Checkbias_list(bias_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	68 bias_list = map(T.as_tensor_variable, bias_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	69 for i in range(len(bias_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	70 nbias = bias_list[i].type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	71 if nbias != 1: raise TypeError('not vector', bias_list[i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	72 return bias_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	73
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	74 # Encoding scan dot product------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	75 class ScanDotEnc(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	76 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	77 the available inputs (as theano.generic), and all the encoding weights tensor.dmatrix of the model. It will select the
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	78 weights corresponding to the inputs (according to index list) and compute only the necessary dot products"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	79 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	80 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	81 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	82 inputs = T.dmatrix('input')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	83 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	84 self.M.hid = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	85 self.M.resultin = self.M.hid + T.dot(inputs,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	86 result = T.dot(inputs,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	87
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	88 self.M.dotin = theano.Method([inputs,weights],None,{self.M.hid : self.M.resultin})
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	89 self.M.dot = theano.Method([inputs,weights],result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	90 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	91
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	92 def make_node(self, idx_list, inputs_list, weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	93 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	94 weights_list = Checkweights_list(weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	95 return Apply(self, [idx_list] + [inputs_list] + weights_list, [T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	96
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	97 def perform(self, node, args, (hid,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	98 idx_list = args[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	99 hidcalc = False
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	100
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	101 batchsize = (args[1][0].shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	102 n_hid = (args[2].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	103 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	104 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	105 if max(idx_list) >= (len(args)-2)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	106 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	107 for i in range(len(args[1])):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	108 if (args[1][i].shape)[0] != batchsize:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	109 raise NotImplementedError('different batchsize in the inputs list',args[1][i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	110 for i in range(len(args)-2):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	111 if (args[2+i].shape)[1] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	112 raise NotImplementedError('different length of hidden in the weights list',args[2+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	113
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	114 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	115 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	116 if hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	117 self.m.dotin(args[1][i],args[2+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	118 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	119 self.m.hid = self.m.dot(args[1][i],args[2+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	120 hidcalc = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	121
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	122 if not hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	123 hid[0] = numpy.zeros([batchsize,n_hid])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	124 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	125 hid[0] = self.m.hid
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	126
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	127
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	128 def grad(self, args, gz):
700 33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	129 gradi = ScanDotEncGrad()(args,gz)
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	130 if type(gradi) != list:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	131 return [None, None] + [gradi]
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	132 else:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	133 return [None, None] + gradi
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	134
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	135 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	136 return hash(ScanDotEnc)^58994
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	137
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	138 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	139 return "ScanDotEnc"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	140
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	141 scandotenc=ScanDotEnc()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	142
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	143 class ScanDotEncGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	144 """This Op computes the gradient wrt the weights for ScanDotEnc"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	145 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	146 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	147 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	148 input1 = T.dmatrix('input1')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	149 self.M.g_out = T.dmatrix('g_out')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	150 result = T.dmatrix('result')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	151 input2=T.transpose(input1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	152 self.M.resultin = result + T.dot(input2,self.M.g_out)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	153 self.M.result = T.dot(input2,self.M.g_out)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	154
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	155 self.M.dotin = theano.Method([input1,result],self.M.resultin)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	156 self.M.dot = theano.Method([input1],self.M.result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	157 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	158
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	159 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	160 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	161 weights_list = Checkweights_list(args[2:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	162 return Apply(self, args + g_out, [T.dmatrix() for i in xrange(2,len(args))])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	163
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	164 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	165 idx_list = args[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	166 self.m.g_out = args[-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	167
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	168 batchsize = (args[1][0].shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	169 n_hid = (args[2].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	170 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	171 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	172 if max(idx_list) >= (len(args)-3)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	173 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	174 for i in range(len(args[1])):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	175 if (args[1][i].shape)[0] != batchsize:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	176 raise NotImplementedError('different batchsize in the inputs list',args[1][i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	177 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	178 if (args[2+i].shape)[1] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	179 raise NotImplementedError('different length of hidden in the weights list',args[2+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	180
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	181 zcalc = [False for i in range(len(args)-3)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	182
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	183 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	184 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	185 if zcalc[int(idx_list[i]-1)]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	186 z[int(idx_list[i]-1)][0] = self.m.dotin(args[1][i],z[int(idx_list[i]-1)][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	187 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	188 z[int(idx_list[i]-1)][0] = self.m.dot(args[1][i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	189 zcalc[int(idx_list[i]-1)] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	190
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	191 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	192 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	193 shp = args[2+i].shape
700 33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	194 z[i][0] = numpy.zeros(shp)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	195
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	196 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	197 return hash(ScanDotEncGrad)^15684
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	198
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	199 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	200 return "ScanDotEncGrad"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	201
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	202 # Decoding scan dot product------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	203 class ScanDotDec(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	204 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	205 the available inputs (as theano.generic), the hidden layer of the DAA (theano.dmatrix)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	206 and all the decoding weights tensor.dmatrix of the model. It will select the
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	207 weights corresponding to the available inputs (according to index list) and compute
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	208 only the necessary dot products. The outputs will be concatenated and will represent
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	209 the reconstruction of the different modality in the same order than the index list"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	210 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	211 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	212 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	213 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	214 self.M.hid = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	215 oldval = T.dmatrix('oldval')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	216 resultin = oldval + T.dot(self.M.hid,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	217 result = T.dot(self.M.hid,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	218
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	219 self.M.dotin = theano.Method([weights,oldval],resultin)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	220 self.M.dot = theano.Method([weights],result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	221 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	222
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	223 def make_node(self, idx_list, input_list, hidd, weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	224 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	225 hidd = Checkhidd(hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	226 weights_list = Checkweights_list(weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	227 return Apply(self, [idx_list] + [input_list] +[hidd] + weights_list,[T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	228
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	229 def perform(self, node, args, (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	230
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	231 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	232 self.m.hid = args[2]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	233
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	234 batchsize = (self.m.hid.shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	235 n_hid = self.m.hid.shape[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	236 if max(idx_list) >= len(args)-3+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	237 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	238 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	239 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	240 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	241 if (args[3+i].shape)[0] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	242 raise NotImplementedError('different length of hidden in the weights list',args[3+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	243
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	244 zcalc = [False for i in idx_list]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	245 z[0] = [None for i in idx_list]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	246
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	247 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	248 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	249 if zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	250 z[0][i] = self.m.dotin(args[3+int(idx_list[i]-1)],z[0][i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	251 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	252 z[0][i] = self.m.dot(args[3+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	253 zcalc[i] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	254
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	255 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	256 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	257 shp = args[1][int(idx_list[i]-1)].shape
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	258 z[0][i] = numpy.zeros((batchsize,shp[1]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	259
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	260 z[0] = numpy.concatenate(z[0],1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	261
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	262 def grad(self, args, gz):
700 33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	263 gradi = ScanDotDecGrad()(args,gz)
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	264 if type(gradi) != list:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	265 return [None, None] + [gradi]
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	266 else:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	267 return [None, None] + gradi
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	268
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	269 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	270 return hash(ScanDotDec)^73568
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	271
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	272 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	273 return "ScanDotDec"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	274
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	275 scandotdec=ScanDotDec()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	276
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	277 class ScanDotDecGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	278 """This Op computes the gradient wrt the weights for ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	279 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	280 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	281 gout = T.dmatrix('gout')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	282 self.M.hidt = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	283 oldval = T.dmatrix('oldval')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	284 resultin1 = oldval + T.dot(self.M.hidt,gout)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	285 result1 = T.dot(self.M.hidt,gout)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	286 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	287 weights2 = T.transpose(weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	288 resultin2 = oldval + T.dot(gout,weights2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	289 result2 = T.dot(gout,weights2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	290
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	291 self.M.dotin1 = theano.Method([gout,oldval],resultin1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	292 self.M.dot1 = theano.Method([gout],result1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	293 self.M.dotin2 = theano.Method([gout,weights,oldval],resultin2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	294 self.M.dot2 = theano.Method([gout,weights],result2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	295 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	296
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	297
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	298 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	299 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	300 hidd = Checkhidd(args[2])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	301 weights_list = Checkweights_list(args[3:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	302 return Apply(self, args + g_out, [T.dmatrix() for i in xrange(2,len(args))])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	303
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	304 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	305 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	306 self.m.hidt = args[2].T
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	307
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	308 batchsize = (self.m.hidt.shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	309 n_hid = self.m.hidt.shape[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	310 if max(idx_list) >= len(args)-4+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	311 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	312 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	313 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	314 for i in range(len(args)-4):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	315 if (args[3+i].shape)[0] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	316 raise NotImplementedError('different length of hidden in the weights list',args[3+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	317
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	318 zidx=numpy.zeros((len(idx_list)+1))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	319
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	320 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	321 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	322 zidx[i+1] = (args[1][i].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	323 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	324 zidx[i+1] = (args[3+idx_list[i]-1].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	325
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	326 zidx=zidx.cumsum()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	327 hidcalc = False
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	328 zcalc = [False for i in range((len(args)-4))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	329
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	330 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	331 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	332 if zcalc[int(idx_list[i])-1]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	333 z[int(idx_list[i])][0] = self.m.dotin1(args[-1][:,zidx[i]:zidx[i+1]],z[int(idx_list[i])][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	334 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	335 z[int(idx_list[i])][0] = self.m.dot1(args[-1][:,zidx[i]:zidx[i+1]])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	336 zcalc[int(idx_list[i])-1] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	337 if hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	338 z[0][0] = self.m.dotin2(args[-1][:,zidx[i]:zidx[i+1]],args[3+int(idx_list[i]-1)],z[0][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	339 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	340 z[0][0] = self.m.dot2(args[-1][:,zidx[i]:zidx[i+1]],args[3+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	341 hidcalc = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	342
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	343 if not hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	344 z[0][0] = numpy.zeros((self.m.hidt.shape[1],self.m.hidt.shape[0]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	345
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	346 for i in range((len(args)-4)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	347 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	348 shp = args[3+i].shape
701 113946723973 fixed bug of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 700 diff changeset	349 z[i+1][0] = numpy.zeros(shp)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	350
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	351
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	352 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	353 return hash(ScanDotDecGrad)^87445
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	354
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	355 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	356 return "ScanDotDecGrad"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	357
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	358 # DAA input noise------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	359 class ScanNoise(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	360 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	361 the available inputs (as theano.generic), a probability of individual bit masking and
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	362 a probability of modality masking. It will return the inputs list with randoms zeros entry
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	363 and the index list with some positive values changed to negative values (groups masking)"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	364 def __init__(self, seed = 1):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	365 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	366 self.M.rand = T.RandomStreams(seed)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	367 self.seed = seed
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	368 mat = T.matrix('mat')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	369 noise_level_bit = T.dscalar('noise_level_bit')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	370 noise_level_group = T.dscalar('noise_level_group')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	371 self.M.out1 = self.M.rand.binomial(T.shape(mat), 1, 1 - noise_level_bit) * mat
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	372 self.M.out2 = self.M.rand.binomial((1,1), 1, 1 - noise_level_group)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	373
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	374 self.M.noisify_bit = theano.Method([mat,noise_level_bit],self.M.out1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	375 self.M.noisify_group_bool = theano.Method([noise_level_group],self.M.out2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	376 self.R = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	377 self.R.rand.initialize()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	378
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	379 def make_node(self, idx_list, inputs_list, noise_level_bit, noise_level_group):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	380 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	381 return Apply(self, [idx_list] + [inputs_list] + [noise_level_bit] + [noise_level_group],\
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	382 [T.ivector(), theano.generic()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	383
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	384 def perform(self, node, (idx_list,inputs_list,noise_level_bit,noise_level_group), (y,z)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	385
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	386 if len(idx_list) != len(inputs_list) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	387 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	388
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	389 y[0] = numpy.asarray([-i if (i>0 and not(self.R.noisify_group_bool(noise_level_group))) else i for i in idx_list])
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	390 z[0] = [(self.R.noisify_bit(inputs_list[i],noise_level_bit) if y[0][i]>0 else numpy.zeros((inputs_list[i].shape)))\
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	391 for i in range(len(inputs_list))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	392
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	393 def grad(self,args,gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	394 return [None,None,None,None]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	395
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	396
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	397 def __hash__(self):
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	398 return hash(ScanNoise)^hash(self.seed)^hash(self.R.rand)^12254
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	399
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	400 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	401 return "ScanNoise"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	402
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	403 scannoise=ScanNoise()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	404
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	405 # Total input matrix construction------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	406 class ScanInputs(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	407 """This Op takes an index list (as tensor.ivector) and a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	408 the available inputs (as theano.generic). It will construct the appropriate tensor.dmatrix
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	409 to compare to the reconstruction obtained with ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	410 def make_node(self, idx_list, inputs_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	411 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	412 return Apply(self, [idx_list] + [inputs_list],[T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	413
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	414 def perform(self, node, (idx_list, inputs_list), (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	415
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	416 if len(idx_list) != len(inputs_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	417 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	418
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	419 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	420 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	421 inputs_list[i] = 0 * inputs_list[i]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	422
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	423 z[0] = numpy.concatenate(inputs_list,1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	424
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	425 def grad(self,args,gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	426 return [None,None]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	427
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	428 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	429 return hash(ScanInputs)^75902
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	430
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	431 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	432 return "ScanInputs"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	433
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	434 scaninputs=ScanInputs()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	435
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	436 # Decoding bias vector construction------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	437 class ScanBiasDec(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	438 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	439 the available inputs (as theano.generic) and the decoding bias tensor.dvector.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	440 It will construct the appropriate bias tensor.dvector
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	441 to add to the reconstruction obtained with ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	442 def make_node(self, idx_list, input_list, bias_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	443 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	444 bias_list = Checkbias_list(bias_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	445 return Apply(self, [idx_list] + [input_list] + bias_list, [T.dvector()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	446
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	447 def perform(self, node, args, (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	448 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	449
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	450 if max(idx_list) >= (len(args)-2)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	451 raise NotImplementedError('index superior to bias list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	452 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	453 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	454 z[0] = [args[idx_list[i]+1] if idx_list[i] != 0 else numpy.zeros(args[1][i].shape[1]) \
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	455 for i in range(len(idx_list))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	456 z[0] = numpy.concatenate(z[0],1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	457
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	458 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	459 return hash(ScanBiasDec)^60056
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	460
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	461 def grad(self,args,gz):
700 33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	462 gradi = ScanBiasDecGrad()(args,gz)
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	463 if type(gradi) != list:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	464 return [None, None] + [gradi]
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	465 else:
33e13d8ca7d3 improved efficiency of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 694 diff changeset	466 return [None, None] + gradi
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	467
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	468 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	469 return "ScanBiasDec"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	470
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	471 scanbiasdec=ScanBiasDec()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	472
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	473 class ScanBiasDecGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	474 """This Op computes the gradient wrt the bias for ScanBiasDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	475 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	476 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	477 bias_list = Checkbias_list(args[2:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	478 return Apply(self, args + g_out, [T.dvector() for i in range(len(args)-2)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	479
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	480 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	481 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	482
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	483 if max(idx_list) >= (len(args)-3)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	484 raise NotImplementedError('index superior to bias list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	485 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	486 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	487
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	488 zidx=numpy.zeros((len(idx_list)+1))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	489 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	490 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	491 zidx[i+1] = (args[1][i].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	492 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	493 zidx[i+1] = (args[2+idx_list[i]-1].size)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	494 zidx=zidx.cumsum()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	495 zcalc = [False for i in range((len(args)-3))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	496
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	497 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	498 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	499 if zcalc[int(idx_list[i])-1]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	500 z[int(idx_list[i])-1][0] += args[-1][zidx[i]:zidx[i+1]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	501 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	502 z[int(idx_list[i])-1][0] = args[-1][zidx[i]:zidx[i+1]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	503 zcalc[int(idx_list[i])-1] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	504
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	505 for i in range((len(args)-3)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	506 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	507 shp = args[2+i].size
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	508 z[i][0] = numpy.zeros(shp)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	509
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	510
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	511 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	512 return hash(ScanBiasDecGrad)^41256
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	513
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	514 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	515 return "ScanBiasDecGrad"
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	516
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	517 # Mask construction------------------------------------
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	518 class ScanMask(Op):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	519 """This Op takes an index list (as tensor.ivector) and a list of weigths.
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	520 It will construct a list of T.iscalar representing the Mask
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	521 to do the correct regularisation on the weigths"""
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	522 def __init__(self,encbool=True):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	523 self.encbool = encbool
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	524
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	525 def make_node(self, idx_list, weights_list):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	526 idx_list = Checkidx_list(idx_list)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	527 weights_list = Checkweights_list(weights_list)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	528 return Apply(self, [idx_list] + weights_list, [T.iscalar() for i in range(len(weights_list))])
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	529
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	530 def perform(self, node, args, z):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	531 if self.encbool:
701 113946723973 fixed bug of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 700 diff changeset	532 idx_list = args[0]
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	533 dim = 1
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	534 else:
701 113946723973 fixed bug of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 700 diff changeset	535 idx_list = abs(args[0])
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	536 dim = 0
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	537 n_hid = args[1].shape[dim]
701 113946723973 fixed bug of scan_inputs_groups Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 700 diff changeset	538
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	539 if max(idx_list) >= (len(args)-1)+1 :
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	540 raise NotImplementedError('index superior to weights list length',idx_listdec)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	541 for i in range(len(args)-1):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	542 if args[1+i].shape[dim] != n_hid:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	543 raise NotImplementedError('different length of hidden in the encoding weights list',args[1+i].shape)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	544
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	545 for i in range(len(args[1:])):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	546 z[i][0] = numpy.asarray((idx_list == i+1).sum(),dtype='int32')
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	547
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	548 def __hash__(self):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	549 return hash(ScanMask)^hash(self.encbool)^11447
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	550
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	551 def grad(self,args,gz):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	552 return [None] * len(args)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	553
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	554 def __str__(self):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	555 if self.encbool:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	556 string = "Enc"
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	557 else:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	558 string = "Dec"
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	559 return "ScanMask" + string
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	560
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	561 scanmaskenc=ScanMask(True)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	562 scanmaskdec=ScanMask(False)

Mercurial > pylearn

annotate pylearn/sandbox/scan_inputs_groups.py @ 701:113946723973