Mercurial > pylearn

annotate sandbox/simple_autoassociator/model.py @ 416:8849eba55520

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Can now do minibatch update
author Joseph Turian <turian@iro.umontreal.ca>
date Fri, 11 Jul 2008 16:34:46 -0400
parents faffaae0d2f9
children 4f61201fa9a9
rev   line source
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
1 """
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
2 The model for an autoassociator for sparse inputs, using Ronan Collobert + Jason
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
3 Weston's sampling trick (2008).
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
4 """
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
5
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
6 from graph import trainfn
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
7 import parameters
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
8
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
9 import globals
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
10 from globals import LR
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
11
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
12 import numpy
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
13 import random
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
14 random.seed(globals.SEED)
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
15
416
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
16 import pylearn.sparse_instance
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
17
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
18 class Model:
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
19 def __init__(self):
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
20 self.parameters = parameters.Parameters(randomly_initialize=True)
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
21
416
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
22 # def deterministic_reconstruction(self, x):
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
23 # (y, h, loss, gw1, gb1, gw2, gb2) = trainfn(x, self.parameters.w1, self.parameters.b1, self.parameters.w2, self.parameters.b2)
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
24 # return y
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
25
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
26 def update(self, instances):
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
27 """
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
28 Update the L{Model} using one training instance.
416
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
29 @param instances: A list of dict from feature index to (non-zero) value.
372
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
30 @todo: Should assert that nonzero_indices and zero_indices
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
31 are correct (i.e. are truly nonzero/zero).
75bab24bb2d8 Moved more logic into model.py
Joseph Turian <turian@gmail.com>
parents: 370
diff changeset
32 """
416
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
33 minibatch = len(instances)
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
34 # x = pylearn.sparse_instance.to_vector(instances, self.input_dimension)
8849eba55520 Can now do minibatch update
Joseph Turian <turian@iro.umontreal.ca>
parents: 411
diff changeset
35 x = pylearn.sparse_instance.to_vector(instances, globals.INPUT_DIMENSION)
386
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 373
diff changeset
36
411
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
37 (y, h, loss, gw1, gb1, gw2, gb2) = trainfn(x, self.parameters.w1, self.parameters.b1, self.parameters.w2, self.parameters.b2)
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
38 # print
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
39 # print "instance:", instance
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
40 # print "x:", x
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
41 # print "OLD y:", y
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
42 print "OLD total loss:", loss
411
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
43 # print "gw1:", gw1
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
44 # print "gb1:", gb1
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
45 # print "gw2:", gw2
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
46 # print "gb2:", gb2
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
47
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
48 # SGD update
386
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 373
diff changeset
49 self.parameters.w1 -= LR * gw1
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 373
diff changeset
50 self.parameters.b1 -= LR * gb1
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 373
diff changeset
51 self.parameters.w2 -= LR * gw2
a474341861fa Added a simple AA
Joseph Turian <turian@gmail.com>
parents: 373
diff changeset
52 self.parameters.b2 -= LR * gb2
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
53
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
54 # Recompute the loss, to make sure it's descreasing
411
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
55 (y, h, loss, gw1, gb1, gw2, gb2) = trainfn(x, self.parameters.w1, self.parameters.b1, self.parameters.w2, self.parameters.b2)
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
56 # print "NEW y:", y
370
a1bbcde6b456 Moved sparse_random_autoassociator from my repository
Joseph Turian <turian@gmail.com>
parents:
diff changeset
57 print "NEW total loss:", loss
411
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
58 # print "h:", h
faffaae0d2f9 Autoassociator now seems to work
Joseph Turian <turian@iro.umontreal.ca>
parents: 404
diff changeset
59 # print self.parameters