Mercurial > ift6266
diff deep/stacked_dae/stacked_dae.py @ 204:e1f5f66dd7dd
Changé le coût de reconstruction pour stabilité numérique, en ajoutant une petite constante dans le log.
| author | fsavard |
|---|---|
| date | Thu, 04 Mar 2010 08:18:42 -0500 |
| parents | e656edaedb48 |
| children | acb942530923 |
line wrap: on
line diff
--- a/deep/stacked_dae/stacked_dae.py Wed Mar 03 12:51:40 2010 -0500 +++ b/deep/stacked_dae/stacked_dae.py Thu Mar 04 08:18:42 2010 -0500 @@ -138,7 +138,15 @@ # note : we sum over the size of a datapoint; if we are using minibatches, # L will be a vector, with one entry per example in minibatch #self.L = - T.sum( self.x*T.log(self.z) + (1-self.x)*T.log(1-self.z), axis=1 ) - self.L = binary_cross_entropy(target=self.x, output=self.z, sum_axis=1) + #self.L = binary_cross_entropy(target=self.x, output=self.z, sum_axis=1) + + # I added this epsilon to avoid getting log(0) and 1/0 in grad + # This means conceptually that there'd be no probability of 0, but that + # doesn't seem to me as important (maybe I'm wrong?). + eps = 0.00000001 + eps_1 = 1-eps + self.L = - T.sum( self.x * T.log(eps + eps_1*self.z) \ + + (1-self.x)*T.log(eps + eps_1*(1-self.z)), axis=1 ) # note : L is now a vector, where each element is the cross-entropy cost # of the reconstruction of the corresponding example of the # minibatch. We need to compute the average of all these to get