Mercurial > pylearn
annotate cost.py @ 484:3daabc7f94ff
Added Yoshua's explanation
| author | Joseph Turian <turian@gmail.com> |
|---|---|
| date | Tue, 28 Oct 2008 01:33:27 -0400 |
| parents | d99fefbc9324 |
| children | 94a4c5b7293b |
| rev | line source |
|---|---|
| 413 | 1 """ |
| 2 Cost functions. | |
| 439 | 3 |
| 4 @note: All of these functions return one cost per example. So it is your | |
| 5 job to perform a tensor.sum over the individual example losses. | |
|
484
3daabc7f94ff
Added Yoshua's explanation
Joseph Turian <turian@gmail.com>
parents:
451
diff
changeset
|
6 |
|
3daabc7f94ff
Added Yoshua's explanation
Joseph Turian <turian@gmail.com>
parents:
451
diff
changeset
|
7 @todo: It would be nice to implement a hinge loss, with a particular margin. |
| 413 | 8 """ |
| 9 | |
| 415 | 10 import theano.tensor as T |
| 451 | 11 from xlogx import xlogx |
| 415 | 12 |
| 413 | 13 def quadratic(target, output, axis=1): |
| 14 return T.mean(T.sqr(target - output), axis) | |
| 15 | |
| 16 def cross_entropy(target, output, axis=1): | |
| 448 | 17 """ |
| 18 @todo: This is essentially duplicated as nnet_ops.binary_crossentropy | |
| 449 | 19 @warning: OUTPUT and TARGET are reversed in nnet_ops.binary_crossentropy |
| 448 | 20 """ |
|
434
0f366ecb11ee
log2->log in cost
Olivier Breuleux <breuleuo@iro.umontreal.ca>
parents:
415
diff
changeset
|
21 return -T.mean(target * T.log(output) + (1 - target) * T.log(1 - output), axis=axis) |
| 451 | 22 |
| 23 def KL_divergence(target, output): | |
| 24 """ | |
| 25 @note: We do not compute the mean, because if target and output have | |
| 26 different shapes then the result will be garbled. | |
| 27 """ | |
| 28 return -(target * T.log(output) + (1 - target) * T.log(1 - output)) \ | |
| 29 + (xlogx(target) + xlogx(1 - target)) | |
| 30 # return cross_entropy(target, output, axis) - cross_entropy(target, target, axis) |