Mercurial > pylearn
view doc/v2_planning/optimization.txt @ 1042:4eaf576c3e9a
dumi's choices
author | Dumitru Erhan <dumitru.erhan@gmail.com> |
---|---|
date | Wed, 08 Sep 2010 11:18:00 -0400 |
parents | 89e76e6e074f |
children | baf1988db557 |
line wrap: on
line source
Discussion of Optimization-Related Issues ========================================= Members: JB, PL, OD, XG Representative: JB Previous work - scikits, openopt, scipy provide function optimization algorithms. These are not currently GPU-enabled but may be in the future. IS PREVIOUS WORK SUFFICIENT? -------------------------------- In many cases it is (I used it for sparse coding, and it was ok). These packages provide batch optimization, whereas we typically need online optimization. It can be faster (to run) and more convenient (to implement) to have optimization algorithms as Theano update expressions. What optimization algorithms do we want/need? --------------------------------------------- - sgd - sgd + momentum - sgd with annealing schedule - TONGA - James Marten's Hessian-free - Conjugate gradients, batch and (large) mini-batch [that is also what Marten's thing does] Do we need anything to make batch algos work better with Pylearn things? - conjugate methods? yes - L-BFGS? maybe, when needed