view pylearn/gd/README.txt @ 1268:78a09cdd449b

added some wishlist items for gd module
author James Bergstra <bergstrj@iro.umontreal.ca>
date Fri, 03 Sep 2010 12:35:49 -0400
parents 5f80351bc762
children
line wrap: on
line source


see __init__.py


TODO: - add an sgd with annealed learning rate (the annealing schedule is implemented via
shared variables and an update schedule

Wishlist:
    - sgd 
    - momentum
    - annealing schedule
    - conjugate methods
    - l-bfgs
    - hessian free
    - tonga
    - smth by nick schraudolff (sp?)

Theano needs lazy if for these to be efficient


Early stopping heuristics as update expressions too?