Mercurial > pylearn
comparison doc/v2_planning/formulas.txt @ 1165:42ddbefd1e03
made the API_formulas.txt and removed duplicate stuff from the formulas.txt file
| author | Frederic Bastien <nouiz@nouiz.org> |
|---|---|
| date | Fri, 17 Sep 2010 13:57:07 -0400 |
| parents | bc246542d6ff |
| children |
comparison
equal
deleted
inserted
replaced
| 1164:53d11eafdaa9 | 1165:42ddbefd1e03 |
|---|---|
| 7 - Razvan | 7 - Razvan |
| 8 - Aaron | 8 - Aaron |
| 9 - Olivier B. | 9 - Olivier B. |
| 10 - Nicolas | 10 - Nicolas |
| 11 | 11 |
| 12 TODO | |
| 13 ---- | |
| 14 * define a list of search tag to start with | |
| 15 * propose an interface(many inputs, outputs, doc style, hierrache, to search, html output?) | |
| 16 * find existing repositories with files for formulas. | |
| 17 * move existing formulas to pylearn as examples and add other basics ones. | |
| 18 ** theano.tensor.nnet will probably be copied to pylearn.formulas.nnet and depricated. | |
| 19 | |
| 20 Why we need formulas | |
| 21 -------------------- | |
| 22 | |
| 23 Their is a few reasons why having a library of mathematical formula for theano is a good reason: | |
| 24 | |
| 25 * Some formula have some special thing needed for the gpu. | |
| 26 * Sometimes we need to cast to floatX... | |
| 27 * Some formula have numerical stability problem. | |
| 28 * Some formula gradiant have numerical stability problem. (Happen more frequently then the previous ones) | |
| 29 * If theano don't always do some stability optimization, we could do it manually in the formulas | |
| 30 * Some formula as complex to implement and take many try to do correctly. | |
| 31 | |
| 32 Having a library help in that we solve those problem only once. | |
| 33 | |
| 34 Formulas definition | |
| 35 ------------------- | |
| 36 | |
| 37 We define formulas as something that don't have a state. They are implemented as python function | |
| 38 that take theano variable as input and output theano variable. If you want state, look at what the | |
| 39 learner commity will do. | |
| 40 | |
| 41 Formulas doc must have | |
| 42 ---------------------- | |
| 43 | |
| 44 * A latex mathematical description of the formulas(for picture representation in generated documentation) | |
| 45 * Tags(for searching): | |
| 46 * a list of lower lovel fct used | |
| 47 * category(name of the submodule itself) | |
| 48 * Tell if we did some work to make it more numerical stable. Do theano do the optimization needed? | |
| 49 * Tell if the grad is numericaly stable? Do theano do the optimization needed? | |
| 50 * Tell if work on gpu/not/unknow | |
| 51 * Tell alternate name | |
| 52 * Tell the domaine, range of the input/output(range should use the english notation of including or excluding) | |
| 53 | 12 |
| 54 List of existing repos | 13 List of existing repos |
| 55 ---------------------- | 14 ---------------------- |
| 56 | 15 |
| 57 Olivier B. ? | 16 Olivier B. ? |
| 58 Xavier G.: git@github.com:glorotxa/DeepANN.git, see file deepANN/{Activations.py(to nnet),Noise.py,Reconstruction_cost.py(to costs),Regularization.py(to regularization} | 17 Xavier G.: git@github.com:glorotxa/DeepANN.git, see file deepANN/{Activations.py(to nnet),Noise.py,Reconstruction_cost.py(to costs),Regularization.py(to regularization} |
| 59 | 18 |
| 60 Proposed hierarchy | |
| 61 ------------------ | |
| 62 | |
| 63 Here is the proposed hierarchy for formulas | |
| 64 | |
| 65 pylearn.formulas.costs: generic / common cost functions, e.g. various cross-entropies, squared error, | |
| 66 abs. error, various sparsity penalties (L1, Student) | |
| 67 | |
| 68 pylearn.formulas.regularization: formulas for regularization | |
| 69 | |
| 70 pylearn.formulas.linear: formulas for linear classifier, linear regression, factor analysis, PCA | |
| 71 | |
| 72 pylearn.formulas.nnet: formulas for building layers of various kinds, various activation functions, | |
| 73 layers which could be plugged with various costs & penalties, and stacked | |
| 74 | |
| 75 pylearn.formulas.ae: formulas for auto-encoders and denoising auto-encoder variants | |
| 76 | |
| 77 pylearn.formulas.noise: formulas for corruption processes | |
| 78 | |
| 79 pylearn.formulas.rbm: energies, free energies, conditional distributions, Gibbs sampling | |
| 80 | |
| 81 pylearn.formulas.trees: formulas for decision trees | |
| 82 | |
| 83 pylearn.formulas.boosting: formulas for boosting variants | |
| 84 | |
| 85 pylearn.formulas.maths for other math formulas | |
| 86 | |
| 87 pylearn.formulas.scipy.stats: example to implement the same interface as existing lib | |
| 88 | |
| 89 etc. |