Mercurial > pylearn

comparison doc/v2_planning/optimization.txt @ 1071:2cf3ad953bf9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
optimization: Removed duplicated API draft and asked question
author Olivier Delalleau <delallea@iro>
date Fri, 10 Sep 2010 10:29:15 -0400
parents 9fe0f0755b03
children 7c5dc11c850a
comparison
equal deleted inserted replaced
1070:79eb0016f333 1071:2cf3ad953bf9
47 47
48 48
49 Proposal for API 49 Proposal for API
50 ================ 50 ================
51 51
52 Stick to the same style of API that we've used for SGD so far. I think it has 52 See api_optimization.txt.
53 worked well. It takes theano expressions as inputs and returns theano
54 expressions as results. The caller is responsible for building those
55 expressions into a callable function that does the minimization (and other
56 things too maybe).
57 53
58 54 OD: Do we really need a different file? If yes, maybe create a subdirectory to
59 def stochastic_gradientbased_optimization_updates(parameters, cost=None, grads=None, **kwargs): 55 be able to easily find all files related to optimization?
60 """
61 :param parameters: list or tuple of Theano variables (typically shared vars)
62 that we want to optimize iteratively algorithm.
63
64 :param cost: scalar-valued Theano variable that computes noisy estimate of
65 cost (what are the conditions on the noise?). The cost is ignored if
66 grads are given.
67
68 :param grads: list or tuple of Theano variables representing the gradients on
69 the corresponding parameters. These default to tensor.grad(cost,
70 parameters).
71
72 :param kwargs: algorithm-dependent arguments
73
74 :returns: a list of pairs (v, new_v) that indicate the value (new_v) each
75 variable (v) should take in order to carry out the optimization procedure.
76
77 The first section of the return value list corresponds to the terms in
78 `parameters`, and the optimization algorithm can return additional update
79 expression afterward. This list of pairs can be passed directly to the
80 dict() constructor to create a dictionary such that dct[v] == new_v.
81 """
82
83
84 Why not a class interface with an __init__ that takes the kwargs, and an
85 updates() that returns the updates? It would be wrong for auxiliary shared
86 variables to be involved in two updates, so the interface should not encourage
87 separate methods for those two steps.
88
89
90