Mercurial > pylearn

annotate sandbox/simple_autoassociator/main.py @ 402:ffdd2c199f2a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
* Added momentum. * Added deterministic_reconstruction_error.
author Joseph Turian <turian@gmail.com>
date Wed, 09 Jul 2008 15:27:12 -0400
parents 36baeb7125a4
children 8849eba55520
rev   line source
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
1 #!/usr/bin/python
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
2 """
386
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 381
diff changeset
3 A simple autoassociator.
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
4
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
5 The learned model is::
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
6 h = sigmoid(dot(x, w1) + b1)
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
7 y = sigmoid(dot(h, w2) + b2)
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
8
386
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 381
diff changeset
9 Binary xent loss.
381
e4473d9697d7 Added xent loss
Joseph Turian <turian@gmail.com>
parents: 372
diff changeset
10
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
11 LIMITATIONS:
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
12 - Only does pure stochastic gradient (batchsize = 1).
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
13 """
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
14
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
15
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 371
diff changeset
16 import numpy
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
17
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
18 nonzero_instances = []
392
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
19 nonzero_instances.append({0: 1, 1: 1})
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
20 nonzero_instances.append({0: 1, 2: 1})
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
21
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
22 #nonzero_instances.append({1: 0.1, 5: 0.5, 9: 1})
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
23 #nonzero_instances.append({2: 0.3, 5: 0.5, 8: 0.8})
e2cb8d489908 More debugging
Joseph Turian <turian@gmail.com>
parents: 389
diff changeset
24 ##nonzero_instances.append({1: 0.2, 2: 0.3, 5: 0.5})
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
25
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
26 import model
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
27 model = model.Model()
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
28
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
29 for i in xrange(100000):
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
30 # Select an instance
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
31 instance = nonzero_instances[i % len(nonzero_instances)]
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
32
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
33 # SGD update over instance
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 371
diff changeset
34 model.update(instance)