Mercurial > pylearn

comparison doc/v2_planning/formulas.txt @ 1051:bc246542d6ff

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added file for the formulas commitee.
author Frederic Bastien <nouiz@nouiz.org>
date Wed, 08 Sep 2010 15:39:51 -0400
parents
children 42ddbefd1e03
comparison
equal deleted inserted replaced
1050:573363a9b5c7 1051:bc246542d6ff
1 Math formulas
2 =============
3
4 Participants
5 ------------
6 - Fred*
7 - Razvan
8 - Aaron
9 - Olivier B.
10 - Nicolas
11
12 TODO
13 ----
14 * define a list of search tag to start with
15 * propose an interface(many inputs, outputs, doc style, hierrache, to search, html output?)
16 * find existing repositories with files for formulas.
17 * move existing formulas to pylearn as examples and add other basics ones.
18 ** theano.tensor.nnet will probably be copied to pylearn.formulas.nnet and depricated.
19
20 Why we need formulas
21 --------------------
22
23 Their is a few reasons why having a library of mathematical formula for theano is a good reason:
24
25 * Some formula have some special thing needed for the gpu.
26 * Sometimes we need to cast to floatX...
27 * Some formula have numerical stability problem.
28 * Some formula gradiant have numerical stability problem. (Happen more frequently then the previous ones)
29 * If theano don't always do some stability optimization, we could do it manually in the formulas
30 * Some formula as complex to implement and take many try to do correctly.
31
32 Having a library help in that we solve those problem only once.
33
34 Formulas definition
35 -------------------
36
37 We define formulas as something that don't have a state. They are implemented as python function
38 that take theano variable as input and output theano variable. If you want state, look at what the
39 learner commity will do.
40
41 Formulas doc must have
42 ----------------------
43
44 * A latex mathematical description of the formulas(for picture representation in generated documentation)
45 * Tags(for searching):
46 * a list of lower lovel fct used
47 * category(name of the submodule itself)
48 * Tell if we did some work to make it more numerical stable. Do theano do the optimization needed?
49 * Tell if the grad is numericaly stable? Do theano do the optimization needed?
50 * Tell if work on gpu/not/unknow
51 * Tell alternate name
52 * Tell the domaine, range of the input/output(range should use the english notation of including or excluding)
53
54 List of existing repos
55 ----------------------
56
57 Olivier B. ?
58 Xavier G.: git@github.com:glorotxa/DeepANN.git, see file deepANN/{Activations.py(to nnet),Noise.py,Reconstruction_cost.py(to costs),Regularization.py(to regularization}
59
60 Proposed hierarchy
61 ------------------
62
63 Here is the proposed hierarchy for formulas
64
65 pylearn.formulas.costs: generic / common cost functions, e.g. various cross-entropies, squared error,
66 abs. error, various sparsity penalties (L1, Student)
67
68 pylearn.formulas.regularization: formulas for regularization
69
70 pylearn.formulas.linear: formulas for linear classifier, linear regression, factor analysis, PCA
71
72 pylearn.formulas.nnet: formulas for building layers of various kinds, various activation functions,
73 layers which could be plugged with various costs & penalties, and stacked
74
75 pylearn.formulas.ae: formulas for auto-encoders and denoising auto-encoder variants
76
77 pylearn.formulas.noise: formulas for corruption processes
78
79 pylearn.formulas.rbm: energies, free energies, conditional distributions, Gibbs sampling
80
81 pylearn.formulas.trees: formulas for decision trees
82
83 pylearn.formulas.boosting: formulas for boosting variants
84
85 pylearn.formulas.maths for other math formulas
86
87 pylearn.formulas.scipy.stats: example to implement the same interface as existing lib
88
89 etc.