Mercurial > pylearn
annotate cost.py @ 477:8ff412852d66
added sgd
| author | James Bergstra <bergstrj@iro.umontreal.ca> |
|---|---|
| date | Fri, 24 Oct 2008 13:28:00 -0400 |
| parents | d99fefbc9324 |
| children | 3daabc7f94ff |
| rev | line source |
|---|---|
| 413 | 1 """ |
| 2 Cost functions. | |
| 439 | 3 |
| 4 @note: All of these functions return one cost per example. So it is your | |
| 5 job to perform a tensor.sum over the individual example losses. | |
| 413 | 6 """ |
| 7 | |
| 415 | 8 import theano.tensor as T |
| 451 | 9 from xlogx import xlogx |
| 415 | 10 |
| 413 | 11 def quadratic(target, output, axis=1): |
| 12 return T.mean(T.sqr(target - output), axis) | |
| 13 | |
| 14 def cross_entropy(target, output, axis=1): | |
| 448 | 15 """ |
| 16 @todo: This is essentially duplicated as nnet_ops.binary_crossentropy | |
| 449 | 17 @warning: OUTPUT and TARGET are reversed in nnet_ops.binary_crossentropy |
| 448 | 18 """ |
|
434
0f366ecb11ee
log2->log in cost
Olivier Breuleux <breuleuo@iro.umontreal.ca>
parents:
415
diff
changeset
|
19 return -T.mean(target * T.log(output) + (1 - target) * T.log(1 - output), axis=axis) |
| 451 | 20 |
| 21 def KL_divergence(target, output): | |
| 22 """ | |
| 23 @note: We do not compute the mean, because if target and output have | |
| 24 different shapes then the result will be garbled. | |
| 25 """ | |
| 26 return -(target * T.log(output) + (1 - target) * T.log(1 - output)) \ | |
| 27 + (xlogx(target) + xlogx(1 - target)) | |
| 28 # return cross_entropy(target, output, axis) - cross_entropy(target, target, axis) |