pylearn: pylearn/sandbox/scan_inputs

annotate pylearn/sandbox/scan_inputs_groups.py @ 694:69947f4e9c0e

added a Mask creation Op and fixed some bugs

author	Xavier Glorot <glorotxa@iro.umontreal.ca>
date	Thu, 14 May 2009 19:34:21 -0400
parents	0457dfa6fcad
children	33e13d8ca7d3

rev	line source
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	1 import numpy
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	2 import theano
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	3 from theano import tensor as T
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	4 from theano.gof import Op
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	5 from theano.gof import Apply
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	6 from theano import scalar as scal
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	7
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	8 # These Ops allows us to deal with static groups of possibly missing inputs efficiently in the dense DAA framework
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	9 # (for exemple with multimodal data with sometimes entire modality missing).
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	10 # The inputs will be represented with an index list and a theano.generic variable (which will be a list of matrices
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	11 # (numpy array), each element will correspond to an available modality and the index list will indicates the weights
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	12 # associated to it).
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	13 # Exemple of index list: [1, 0, -3]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	14 # *the 1 says that the first element of the input list will refer to the first element of the weights_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	15 # (auxiliary target as input)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	16 # if inputslist[i]>0 it refers to Weightslist[indexlist[i]-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	17 # *the 0 means that the second element of the input list will not be encoded neither decoded (it is remplaced by zeros)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	18 # this is not efficient, so in this case it is better to give: [1,-3] and [inputslist[0],inputslist[2]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	19 # but it allows us to deal with empty lists: give indexlist = [.0] and inputlist=[[.0]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	20 # *when an index is negative it means that the input will not be used for encoding but we will still reconstruct it
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	21 # (auxiliary target as output)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	22 # if inputslist[i]<0 it refers to Weightslist[-indexlist[i]-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	23 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	24 # An entire batch should have the same available inputs configuration.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	25 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	26 # Dense DAA Exemple:----------------------------------------------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	27 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	28 #from theano.tensor.nnet import sigmoid
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	29 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	30 #nb_modality = 4
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	31 #wenc = [T.dmatrix('wenc%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	32 #wdec = [T.dmatrix('wdec%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	33 #benc = T.dvector('benc')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	34 #bdec = [T.dvector('bdec%s'%i) for i in range(nb_modality)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	35 #vectin = T.ivector('vectin')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	36 #inputpart = theano.generic('inputpart')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	37 #noise_bit = T.dscalar('noise_bit')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	38 #noise_group = T.dscalar('noise_group')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	39 #
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	40 #[vectin2,inputpart2] = scannoise(vectin,inputpart,noise_bit,noise_group)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	41 #hid = scandotenc(vectin2, inputpart2, wenc)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	42 #acthid = sigmoid(hid + benc)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	43 #dec = sigmoid(scanbiasdec(vectin2,inputpart2,bdec) + scandotdec(vectin2, inputpart2,acthid,wdec))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	44 #cost = T.sum(T.sum(T.sqr( scaninput(vectin,inputpart) - rec ),1),0)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	45
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	46 # Checking inputs in make_node methods----------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	47 def Checkidx_list(idx_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	48 idx_list = T.as_tensor_variable(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	49 nidx = idx_list.type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	50 if nidx != 1: raise TypeError('not vector', idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	51 return idx_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	52
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	53 def Checkhidd(hidd):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	54 hidd = T.as_tensor_variable(hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	55 nhidd = hidd.type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	56 if nhidd not in (1,2): raise TypeError('not matrix or vector', hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	57 return hidd
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	58
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	59 def Checkweights_list(weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	60 weights_list = map(T.as_tensor_variable, weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	61 for i in range(len(weights_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	62 nweights = weights_list[i].type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	63 if nweights not in (1,2): raise TypeError('not matrix or vector', weights_list[i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	64 return weights_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	65
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	66 def Checkbias_list(bias_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	67 bias_list = map(T.as_tensor_variable, bias_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	68 for i in range(len(bias_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	69 nbias = bias_list[i].type.ndim
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	70 if nbias != 1: raise TypeError('not vector', bias_list[i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	71 return bias_list
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	72
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	73 # Encoding scan dot product------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	74 class ScanDotEnc(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	75 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	76 the available inputs (as theano.generic), and all the encoding weights tensor.dmatrix of the model. It will select the
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	77 weights corresponding to the inputs (according to index list) and compute only the necessary dot products"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	78 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	79 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	80 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	81 inputs = T.dmatrix('input')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	82 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	83 self.M.hid = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	84 self.M.resultin = self.M.hid + T.dot(inputs,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	85 result = T.dot(inputs,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	86
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	87 self.M.dotin = theano.Method([inputs,weights],None,{self.M.hid : self.M.resultin})
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	88 self.M.dot = theano.Method([inputs,weights],result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	89 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	90
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	91 def make_node(self, idx_list, inputs_list, weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	92 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	93 weights_list = Checkweights_list(weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	94 return Apply(self, [idx_list] + [inputs_list] + weights_list, [T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	95
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	96 def perform(self, node, args, (hid,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	97 idx_list = args[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	98 hidcalc = False
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	99
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	100 batchsize = (args[1][0].shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	101 n_hid = (args[2].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	102 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	103 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	104 if max(idx_list) >= (len(args)-2)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	105 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	106 for i in range(len(args[1])):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	107 if (args[1][i].shape)[0] != batchsize:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	108 raise NotImplementedError('different batchsize in the inputs list',args[1][i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	109 for i in range(len(args)-2):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	110 if (args[2+i].shape)[1] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	111 raise NotImplementedError('different length of hidden in the weights list',args[2+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	112
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	113 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	114 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	115 if hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	116 self.m.dotin(args[1][i],args[2+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	117 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	118 self.m.hid = self.m.dot(args[1][i],args[2+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	119 hidcalc = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	120
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	121 if not hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	122 hid[0] = numpy.zeros([batchsize,n_hid])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	123 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	124 hid[0] = self.m.hid
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	125
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	126
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	127 def grad(self, args, gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	128 return [None, None] + ScanDotEncGrad()(args,gz)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	129
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	130 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	131 return hash(ScanDotEnc)^58994
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	132
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	133 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	134 return "ScanDotEnc"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	135
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	136 scandotenc=ScanDotEnc()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	137
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	138 class ScanDotEncGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	139 """This Op computes the gradient wrt the weights for ScanDotEnc"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	140 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	141 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	142 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	143 input1 = T.dmatrix('input1')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	144 self.M.g_out = T.dmatrix('g_out')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	145 result = T.dmatrix('result')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	146 input2=T.transpose(input1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	147 self.M.resultin = result + T.dot(input2,self.M.g_out)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	148 self.M.result = T.dot(input2,self.M.g_out)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	149
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	150 self.M.dotin = theano.Method([input1,result],self.M.resultin)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	151 self.M.dot = theano.Method([input1],self.M.result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	152 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	153
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	154 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	155 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	156 weights_list = Checkweights_list(args[2:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	157 return Apply(self, args + g_out, [T.dmatrix() for i in xrange(2,len(args))])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	158
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	159 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	160 idx_list = args[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	161 self.m.g_out = args[-1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	162
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	163 batchsize = (args[1][0].shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	164 n_hid = (args[2].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	165 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	166 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	167 if max(idx_list) >= (len(args)-3)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	168 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	169 for i in range(len(args[1])):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	170 if (args[1][i].shape)[0] != batchsize:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	171 raise NotImplementedError('different batchsize in the inputs list',args[1][i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	172 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	173 if (args[2+i].shape)[1] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	174 raise NotImplementedError('different length of hidden in the weights list',args[2+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	175
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	176 zcalc = [False for i in range(len(args)-3)]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	177
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	178 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	179 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	180 if zcalc[int(idx_list[i]-1)]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	181 z[int(idx_list[i]-1)][0] = self.m.dotin(args[1][i],z[int(idx_list[i]-1)][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	182 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	183 z[int(idx_list[i]-1)][0] = self.m.dot(args[1][i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	184 zcalc[int(idx_list[i]-1)] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	185
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	186 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	187 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	188 shp = args[2+i].shape
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	189 z[i][0] = numpy.zeros((shp[0],shp[1]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	190
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	191 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	192 return hash(ScanDotEncGrad)^15684
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	193
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	194 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	195 return "ScanDotEncGrad"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	196
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	197 # Decoding scan dot product------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	198 class ScanDotDec(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	199 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	200 the available inputs (as theano.generic), the hidden layer of the DAA (theano.dmatrix)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	201 and all the decoding weights tensor.dmatrix of the model. It will select the
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	202 weights corresponding to the available inputs (according to index list) and compute
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	203 only the necessary dot products. The outputs will be concatenated and will represent
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	204 the reconstruction of the different modality in the same order than the index list"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	205 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	206 #Create Theano methods to do the dot products with blas or at least in C.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	207 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	208 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	209 self.M.hid = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	210 oldval = T.dmatrix('oldval')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	211 resultin = oldval + T.dot(self.M.hid,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	212 result = T.dot(self.M.hid,weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	213
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	214 self.M.dotin = theano.Method([weights,oldval],resultin)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	215 self.M.dot = theano.Method([weights],result)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	216 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	217
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	218 def make_node(self, idx_list, input_list, hidd, weights_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	219 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	220 hidd = Checkhidd(hidd)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	221 weights_list = Checkweights_list(weights_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	222 return Apply(self, [idx_list] + [input_list] +[hidd] + weights_list,[T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	223
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	224 def perform(self, node, args, (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	225
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	226 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	227 self.m.hid = args[2]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	228
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	229 batchsize = (self.m.hid.shape)[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	230 n_hid = self.m.hid.shape[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	231 if max(idx_list) >= len(args)-3+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	232 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	233 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	234 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	235 for i in range(len(args)-3):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	236 if (args[3+i].shape)[0] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	237 raise NotImplementedError('different length of hidden in the weights list',args[3+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	238
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	239 zcalc = [False for i in idx_list]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	240 z[0] = [None for i in idx_list]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	241
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	242 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	243 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	244 if zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	245 z[0][i] = self.m.dotin(args[3+int(idx_list[i]-1)],z[0][i])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	246 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	247 z[0][i] = self.m.dot(args[3+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	248 zcalc[i] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	249
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	250 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	251 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	252 shp = args[1][int(idx_list[i]-1)].shape
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	253 z[0][i] = numpy.zeros((batchsize,shp[1]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	254
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	255 z[0] = numpy.concatenate(z[0],1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	256
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	257 def grad(self, args, gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	258 return [None, None] + ScanDotDecGrad()(args,gz)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	259
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	260 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	261 return hash(ScanDotDec)^73568
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	262
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	263 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	264 return "ScanDotDec"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	265
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	266 scandotdec=ScanDotDec()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	267
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	268 class ScanDotDecGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	269 """This Op computes the gradient wrt the weights for ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	270 def __init__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	271 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	272 gout = T.dmatrix('gout')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	273 self.M.hidt = T.dmatrix('hid')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	274 oldval = T.dmatrix('oldval')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	275 resultin1 = oldval + T.dot(self.M.hidt,gout)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	276 result1 = T.dot(self.M.hidt,gout)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	277 weights = T.dmatrix('weights')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	278 weights2 = T.transpose(weights)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	279 resultin2 = oldval + T.dot(gout,weights2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	280 result2 = T.dot(gout,weights2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	281
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	282 self.M.dotin1 = theano.Method([gout,oldval],resultin1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	283 self.M.dot1 = theano.Method([gout],result1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	284 self.M.dotin2 = theano.Method([gout,weights,oldval],resultin2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	285 self.M.dot2 = theano.Method([gout,weights],result2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	286 self.m = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	287
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	288
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	289 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	290 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	291 hidd = Checkhidd(args[2])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	292 weights_list = Checkweights_list(args[3:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	293 return Apply(self, args + g_out, [T.dmatrix() for i in xrange(2,len(args))])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	294
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	295 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	296 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	297 self.m.hidt = args[2].T
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	298
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	299 batchsize = (self.m.hidt.shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	300 n_hid = self.m.hidt.shape[0]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	301 if max(idx_list) >= len(args)-4+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	302 raise NotImplementedError('index superior to weight list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	303 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	304 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	305 for i in range(len(args)-4):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	306 if (args[3+i].shape)[0] != n_hid:
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	307 raise NotImplementedError('different length of hidden in the weights list',args[3+i].shape)
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	308
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	309 zidx=numpy.zeros((len(idx_list)+1))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	310
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	311 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	312 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	313 zidx[i+1] = (args[1][i].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	314 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	315 zidx[i+1] = (args[3+idx_list[i]-1].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	316
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	317 zidx=zidx.cumsum()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	318 hidcalc = False
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	319 zcalc = [False for i in range((len(args)-4))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	320
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	321 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	322 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	323 if zcalc[int(idx_list[i])-1]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	324 z[int(idx_list[i])][0] = self.m.dotin1(args[-1][:,zidx[i]:zidx[i+1]],z[int(idx_list[i])][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	325 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	326 z[int(idx_list[i])][0] = self.m.dot1(args[-1][:,zidx[i]:zidx[i+1]])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	327 zcalc[int(idx_list[i])-1] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	328 if hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	329 z[0][0] = self.m.dotin2(args[-1][:,zidx[i]:zidx[i+1]],args[3+int(idx_list[i]-1)],z[0][0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	330 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	331 z[0][0] = self.m.dot2(args[-1][:,zidx[i]:zidx[i+1]],args[3+int(idx_list[i]-1)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	332 hidcalc = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	333
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	334 if not hidcalc:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	335 z[0][0] = numpy.zeros((self.m.hidt.shape[1],self.m.hidt.shape[0]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	336
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	337 for i in range((len(args)-4)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	338 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	339 shp = args[3+i].shape
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	340 z[i+1][0] = numpy.zeros((shp[0],shp[1]))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	341
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	342
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	343 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	344 return hash(ScanDotDecGrad)^87445
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	345
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	346 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	347 return "ScanDotDecGrad"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	348
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	349 # DAA input noise------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	350 class ScanNoise(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	351 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	352 the available inputs (as theano.generic), a probability of individual bit masking and
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	353 a probability of modality masking. It will return the inputs list with randoms zeros entry
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	354 and the index list with some positive values changed to negative values (groups masking)"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	355 def __init__(self, seed = 1):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	356 self.M=theano.Module()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	357 self.M.rand = T.RandomStreams(seed)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	358 self.seed = seed
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	359 mat = T.matrix('mat')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	360 noise_level_bit = T.dscalar('noise_level_bit')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	361 noise_level_group = T.dscalar('noise_level_group')
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	362 self.M.out1 = self.M.rand.binomial(T.shape(mat), 1, 1 - noise_level_bit) * mat
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	363 self.M.out2 = self.M.rand.binomial((1,1), 1, 1 - noise_level_group)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	364
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	365 self.M.noisify_bit = theano.Method([mat,noise_level_bit],self.M.out1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	366 self.M.noisify_group_bool = theano.Method([noise_level_group],self.M.out2)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	367 self.R = self.M.make()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	368 self.R.rand.initialize()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	369
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	370 def make_node(self, idx_list, inputs_list, noise_level_bit, noise_level_group):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	371 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	372 return Apply(self, [idx_list] + [inputs_list] + [noise_level_bit] + [noise_level_group],\
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	373 [T.ivector(), theano.generic()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	374
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	375 def perform(self, node, (idx_list,inputs_list,noise_level_bit,noise_level_group), (y,z)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	376
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	377 if len(idx_list) != len(inputs_list) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	378 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	379
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	380 y[0] = numpy.asarray([-i if (i>0 and not(self.R.noisify_group_bool(noise_level_group))) else i for i in idx_list])
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	381 z[0] = [(self.R.noisify_bit(inputs_list[i],noise_level_bit) if y[0][i]>0 else numpy.zeros((inputs_list[i].shape)))\
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	382 for i in range(len(inputs_list))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	383
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	384 def grad(self,args,gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	385 return [None,None,None,None]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	386
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	387
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	388 def __hash__(self):
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	389 return hash(ScanNoise)^hash(self.seed)^hash(self.R.rand)^12254
686 0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	390
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	391 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	392 return "ScanNoise"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	393
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	394 scannoise=ScanNoise()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	395
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	396 # Total input matrix construction------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	397 class ScanInputs(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	398 """This Op takes an index list (as tensor.ivector) and a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	399 the available inputs (as theano.generic). It will construct the appropriate tensor.dmatrix
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	400 to compare to the reconstruction obtained with ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	401 def make_node(self, idx_list, inputs_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	402 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	403 return Apply(self, [idx_list] + [inputs_list],[T.dmatrix()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	404
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	405 def perform(self, node, (idx_list, inputs_list), (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	406
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	407 if len(idx_list) != len(inputs_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	408 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	409
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	410 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	411 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	412 inputs_list[i] = 0 * inputs_list[i]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	413
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	414 z[0] = numpy.concatenate(inputs_list,1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	415
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	416 def grad(self,args,gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	417 return [None,None]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	418
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	419 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	420 return hash(ScanInputs)^75902
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	421
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	422 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	423 return "ScanInputs"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	424
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	425 scaninputs=ScanInputs()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	426
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	427 # Decoding bias vector construction------------------------------------
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	428 class ScanBiasDec(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	429 """This Op takes an index list (as tensor.ivector), a list of matrices representing
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	430 the available inputs (as theano.generic) and the decoding bias tensor.dvector.
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	431 It will construct the appropriate bias tensor.dvector
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	432 to add to the reconstruction obtained with ScanDotDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	433 def make_node(self, idx_list, input_list, bias_list):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	434 idx_list = Checkidx_list(idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	435 bias_list = Checkbias_list(bias_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	436 return Apply(self, [idx_list] + [input_list] + bias_list, [T.dvector()])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	437
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	438 def perform(self, node, args, (z,)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	439 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	440
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	441 if max(idx_list) >= (len(args)-2)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	442 raise NotImplementedError('index superior to bias list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	443 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	444 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	445 z[0] = [args[idx_list[i]+1] if idx_list[i] != 0 else numpy.zeros(args[1][i].shape[1]) \
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	446 for i in range(len(idx_list))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	447 z[0] = numpy.concatenate(z[0],1)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	448
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	449 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	450 return hash(ScanBiasDec)^60056
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	451
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	452 def grad(self,args,gz):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	453 return [None,None] + ScanBiasDecGrad()(args,gz)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	454
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	455 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	456 return "ScanBiasDec"
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	457
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	458 scanbiasdec=ScanBiasDec()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	459
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	460 class ScanBiasDecGrad(Op):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	461 """This Op computes the gradient wrt the bias for ScanBiasDec"""
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	462 def make_node(self, args, g_out):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	463 idx_list = Checkidx_list(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	464 bias_list = Checkbias_list(args[2:])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	465 return Apply(self, args + g_out, [T.dvector() for i in range(len(args)-2)])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	466
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	467 def perform(self, node, args, z):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	468 idx_list = abs(args[0])
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	469
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	470 if max(idx_list) >= (len(args)-3)+1 :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	471 raise NotImplementedError('index superior to bias list length',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	472 if len(idx_list) != len(args[1]) :
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	473 raise NotImplementedError('size of index different of inputs list size',idx_list)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	474
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	475 zidx=numpy.zeros((len(idx_list)+1))
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	476 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	477 if idx_list[i] == 0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	478 zidx[i+1] = (args[1][i].shape)[1]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	479 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	480 zidx[i+1] = (args[2+idx_list[i]-1].size)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	481 zidx=zidx.cumsum()
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	482 zcalc = [False for i in range((len(args)-3))]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	483
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	484 for i in range(len(idx_list)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	485 if idx_list[i]>0:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	486 if zcalc[int(idx_list[i])-1]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	487 z[int(idx_list[i])-1][0] += args[-1][zidx[i]:zidx[i+1]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	488 else:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	489 z[int(idx_list[i])-1][0] = args[-1][zidx[i]:zidx[i+1]]
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	490 zcalc[int(idx_list[i])-1] = True
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	491
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	492 for i in range((len(args)-3)):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	493 if not zcalc[i]:
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	494 shp = args[2+i].size
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	495 z[i][0] = numpy.zeros(shp)
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	496
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	497
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	498 def __hash__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	499 return hash(ScanBiasDecGrad)^41256
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	500
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	501 def __str__(self):
0457dfa6fcad add direclty the file scan_inputs_groups.py to sandbox and remove the directory input_groups Foo Bar <barfoo@iro.umontreal.ca> parents: diff changeset	502 return "ScanBiasDecGrad"
694 69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	503
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	504 # Mask construction------------------------------------
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	505 class ScanMask(Op):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	506 """This Op takes an index list (as tensor.ivector) and a list of weigths.
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	507 It will construct a list of T.iscalar representing the Mask
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	508 to do the correct regularisation on the weigths"""
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	509 def __init__(self,encbool=True):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	510 self.encbool = encbool
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	511
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	512 def make_node(self, idx_list, weights_list):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	513 idx_list = Checkidx_list(idx_list)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	514 weights_list = Checkweights_list(weights_list)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	515 return Apply(self, [idx_list] + weights_list, [T.iscalar() for i in range(len(weights_list))])
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	516
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	517 def perform(self, node, args, z):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	518 if self.encbool:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	519 idx_list = args[0][args[0]>0]
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	520 dim = 1
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	521 else:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	522 idx_list = abs(args[0][args[0] != 0])
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	523 dim = 0
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	524 n_hid = args[1].shape[dim]
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	525
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	526 if max(idx_list) >= (len(args)-1)+1 :
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	527 raise NotImplementedError('index superior to weights list length',idx_listdec)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	528 for i in range(len(args)-1):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	529 if args[1+i].shape[dim] != n_hid:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	530 raise NotImplementedError('different length of hidden in the encoding weights list',args[1+i].shape)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	531
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	532 for i in range(len(args[1:])):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	533 z[i][0] = numpy.asarray((idx_list == i+1).sum(),dtype='int32')
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	534
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	535 def __hash__(self):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	536 return hash(ScanMask)^hash(self.encbool)^11447
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	537
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	538 def grad(self,args,gz):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	539 return [None] * len(args)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	540
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	541 def __str__(self):
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	542 if self.encbool:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	543 string = "Enc"
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	544 else:
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	545 string = "Dec"
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	546 return "ScanMask" + string
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	547
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	548 scanmaskenc=ScanMask(True)
69947f4e9c0e added a Mask creation Op and fixed some bugs Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 686 diff changeset	549 scanmaskdec=ScanMask(False)

Mercurial > pylearn

annotate pylearn/sandbox/scan_inputs_groups.py @ 694:69947f4e9c0e