Mercurial > pylearn

comparison sparse_random_autoassociator/main.py @ 371:22463a194c90

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Update doc
author Joseph Turian <turian@gmail.com>
date Mon, 07 Jul 2008 01:57:49 -0400
parents a1bbcde6b456
children 75bab24bb2d8
comparison
equal deleted inserted replaced
370:a1bbcde6b456 371:22463a194c90
5 5
6 The learned model is:: 6 The learned model is::
7 h = sigmoid(dot(x, w1) + b1) 7 h = sigmoid(dot(x, w1) + b1)
8 y = sigmoid(dot(h, w2) + b2) 8 y = sigmoid(dot(h, w2) + b2)
9 9
10 We assume that most of the inputs are zero, and hence that we can 10 We assume that most of the inputs are zero, and hence that
11 separate x into xnonzero, x's nonzero components, and a xzero, 11 we can separate x into xnonzero, x's nonzero components, and
12 a sample of the zeros. (We randomly without replacement choose 12 xzero, a sample of the zeros. We sample---randomly without
13 ZERO_SAMPLE_SIZE zero columns.) 13 replacement---ZERO_SAMPLE_SIZE zero columns from x.
14 14
15 The desideratum is that every nonzero entry is separated from every 15 The desideratum is that every nonzero entry is separated from every
16 zero entry by margin at least MARGIN. 16 zero entry by margin at least MARGIN.
17 For each ynonzero, we want it to exceed max(yzero) by at least MARGIN. 17 For each ynonzero, we want it to exceed max(yzero) by at least MARGIN.
18 For each yzero, we want it to be exceed by min(ynonzero) by at least MARGIN. 18 For each yzero, we want it to be exceed by min(ynonzero) by at least MARGIN.