Mercurial > pylearn

annotate algorithms/sgd.py @ 519:b267a8000f92

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added theano.Member to Member variables which were passed in during initialization.
author Joseph Turian <turian@iro.umontreal.ca>
date Fri, 14 Nov 2008 02:07:20 -0500
parents fbfd3932fd00
children
rev   line source
477
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2 from theano.compile import module
479
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 477
diff changeset
3 from theano import tensor as T
477
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
4
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
5 class StochasticGradientDescent(module.FancyModule):
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
6 def __init__(self, params, gparams, lr=None):
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
7 super(StochasticGradientDescent, self).__init__()
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
8
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
9 self.lr = lr if lr is not None else module.Member(T.dscalar())
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
10 self.params = params
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
11 self.gparams = gparams
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
12
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
13 self.updates = dict((p, p - self.lr * g) for p, g in zip(self.params, self.gparams))
8ff412852d66 added sgd
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
14