Mercurial > pylearn
diff doc/v2_planning/optimization.txt @ 1057:baf1988db557
v2planning optimization - added API
author | James Bergstra <bergstrj@iro.umontreal.ca> |
---|---|
date | Thu, 09 Sep 2010 11:32:42 -0400 |
parents | 89e76e6e074f |
children | a41cc29cee26 |
line wrap: on
line diff
--- a/doc/v2_planning/optimization.txt Wed Sep 08 20:45:17 2010 -0400 +++ b/doc/v2_planning/optimization.txt Thu Sep 09 11:32:42 2010 -0400 @@ -39,3 +39,46 @@ + +Proposal for API +================ + +Stick to the same style of API that we've used for SGD so far. I think it has +worked well. It takes theano expressions as inputs and returns theano +expressions as results. The caller is responsible for building those +expressions into a callable function that does the minimization (and other +things too maybe). + + +def stochastic_gradientbased_optimization_updates(parameters, cost=None, grads=None, **kwargs): + """ + :param parameters: list or tuple of Theano variables (typically shared vars) + that we want to optimize iteratively algorithm. + + :param cost: scalar-valued Theano variable that computes noisy estimate of + cost (what are the conditions on the noise?). The cost is ignored if + grads are given. + + :param grads: list or tuple of Theano variables representing the gradients on + the corresponding parameters. These default to tensor.grad(cost, + parameters). + + :param kwargs: algorithm-dependent arguments + + :returns: a list of pairs (v, new_v) that indicate the value (new_v) each + variable (v) should take in order to carry out the optimization procedure. + + The first section of the return value list corresponds to the terms in + `parameters`, and the optimization algorithm can return additional update + expression afterward. This list of pairs can be passed directly to the + dict() constructor to create a dictionary such that dct[v] == new_v. + """ + + +Why not a class interface with an __init__ that takes the kwargs, and an +updates() that returns the updates? It would be wrong for auxiliary shared +variables to be involved in two updates, so the interface should not encourage +separate methods for those two steps. + + +