Mercurial > ift6266

comparison deep/stacked_dae/stacked_dae.py @ 204:e1f5f66dd7dd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Changé le coût de reconstruction pour stabilité numérique, en ajoutant une petite constante dans le log.
author fsavard
date Thu, 04 Mar 2010 08:18:42 -0500
parents e656edaedb48
children acb942530923
comparison
equal deleted inserted replaced
192:e656edaedb48 204:e1f5f66dd7dd
136 self.z = T.nnet.sigmoid(T.dot(self.y, self.W_prime) + self.b_prime) 136 self.z = T.nnet.sigmoid(T.dot(self.y, self.W_prime) + self.b_prime)
137 # Equation (4) 137 # Equation (4)
138 # note : we sum over the size of a datapoint; if we are using minibatches, 138 # note : we sum over the size of a datapoint; if we are using minibatches,
139 # L will be a vector, with one entry per example in minibatch 139 # L will be a vector, with one entry per example in minibatch
140 #self.L = - T.sum( self.x*T.log(self.z) + (1-self.x)*T.log(1-self.z), axis=1 ) 140 #self.L = - T.sum( self.x*T.log(self.z) + (1-self.x)*T.log(1-self.z), axis=1 )
141 self.L = binary_cross_entropy(target=self.x, output=self.z, sum_axis=1) 141 #self.L = binary_cross_entropy(target=self.x, output=self.z, sum_axis=1)
142
143 # I added this epsilon to avoid getting log(0) and 1/0 in grad
144 # This means conceptually that there'd be no probability of 0, but that
145 # doesn't seem to me as important (maybe I'm wrong?).
146 eps = 0.00000001
147 eps_1 = 1-eps
148 self.L = - T.sum( self.x * T.log(eps + eps_1*self.z) \
149 + (1-self.x)*T.log(eps + eps_1*(1-self.z)), axis=1 )
142 # note : L is now a vector, where each element is the cross-entropy cost 150 # note : L is now a vector, where each element is the cross-entropy cost
143 # of the reconstruction of the corresponding example of the 151 # of the reconstruction of the corresponding example of the
144 # minibatch. We need to compute the average of all these to get 152 # minibatch. We need to compute the average of all these to get
145 # the cost of the minibatch 153 # the cost of the minibatch
146 self.cost = T.mean(self.L) 154 self.cost = T.mean(self.L)