Mercurial > pylearn
comparison doc/v2_planning/optimization.txt @ 1057:baf1988db557
v2planning optimization - added API
| author | James Bergstra <bergstrj@iro.umontreal.ca> |
|---|---|
| date | Thu, 09 Sep 2010 11:32:42 -0400 |
| parents | 89e76e6e074f |
| children | a41cc29cee26 |
comparison
equal
deleted
inserted
replaced
| 1055:bc3f7834db83 | 1057:baf1988db557 |
|---|---|
| 37 - L-BFGS? maybe, when needed | 37 - L-BFGS? maybe, when needed |
| 38 | 38 |
| 39 | 39 |
| 40 | 40 |
| 41 | 41 |
| 42 | |
| 43 Proposal for API | |
| 44 ================ | |
| 45 | |
| 46 Stick to the same style of API that we've used for SGD so far. I think it has | |
| 47 worked well. It takes theano expressions as inputs and returns theano | |
| 48 expressions as results. The caller is responsible for building those | |
| 49 expressions into a callable function that does the minimization (and other | |
| 50 things too maybe). | |
| 51 | |
| 52 | |
| 53 def stochastic_gradientbased_optimization_updates(parameters, cost=None, grads=None, **kwargs): | |
| 54 """ | |
| 55 :param parameters: list or tuple of Theano variables (typically shared vars) | |
| 56 that we want to optimize iteratively algorithm. | |
| 57 | |
| 58 :param cost: scalar-valued Theano variable that computes noisy estimate of | |
| 59 cost (what are the conditions on the noise?). The cost is ignored if | |
| 60 grads are given. | |
| 61 | |
| 62 :param grads: list or tuple of Theano variables representing the gradients on | |
| 63 the corresponding parameters. These default to tensor.grad(cost, | |
| 64 parameters). | |
| 65 | |
| 66 :param kwargs: algorithm-dependent arguments | |
| 67 | |
| 68 :returns: a list of pairs (v, new_v) that indicate the value (new_v) each | |
| 69 variable (v) should take in order to carry out the optimization procedure. | |
| 70 | |
| 71 The first section of the return value list corresponds to the terms in | |
| 72 `parameters`, and the optimization algorithm can return additional update | |
| 73 expression afterward. This list of pairs can be passed directly to the | |
| 74 dict() constructor to create a dictionary such that dct[v] == new_v. | |
| 75 """ | |
| 76 | |
| 77 | |
| 78 Why not a class interface with an __init__ that takes the kwargs, and an | |
| 79 updates() that returns the updates? It would be wrong for auxiliary shared | |
| 80 variables to be involved in two updates, so the interface should not encourage | |
| 81 separate methods for those two steps. | |
| 82 | |
| 83 | |
| 84 |