Mercurial > pylearn

annotate doc/v2_planning/sampler.txt @ 1423:ea5d27727869

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added pickle disk usage inspection utility 'pkldu'
author Ian Goodfellow
date Mon, 07 Feb 2011 17:04:56 -0500
parents 0e12ea6ba661
children
rev   line source
1027
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
1
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
2 Inference / Sampling committee: JB, GD, AC
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
3
1018
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
4 OVERVIEW
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
5 ========
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
6
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
7 Before we start defining what a sampler is and how it should be defined in
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
8 pylearn, we should first know what we're up against.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
9
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
10 The workflow I have in mind is the following:
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
11 1. identify the most popular sampling algorithms in the litterature
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
12 2. get up to speed with methods we're not familiar with
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
13 3. identify common usage patterns, properties of the algorithm, etc.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
14 4. decide on an API / best way to implement them
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
15 5. prioritize the algorithms
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
16 6. code away
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
17
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
18 1.BACKGROUND
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
19 =============
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
20
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
21 This section should provide a brief overview of what exists in the litterature.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
22 We should make sure to have a decent understanding of all of these (not everyone
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
23 has to be experts though), so that we can *intelligently* design our sampler
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
24 interface based on common usage patterns, properties, etc.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
25
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
26 Sampling from basic distributions
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
27 * already supported: uniform, normal, binomial, multinomial
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
28 * wish list: beta, poisson, others ?
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
29
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
30 List of sampling algorithms:
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
31 * inversion sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
32 * rejection sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
33 * importance sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
34 * Markov Chain Monte Carlo
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
35 * Gibbs sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
36 * Metropolis Hastings
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
37 * Slice Sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
38 * Annealing
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
39 * Parallel Tempering, Tempered Transitions, Simulated Tempering
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
40 * Nested Sampling (?)
1027
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
41 * Hamiltonian Monte Carlo --> or is it Hybrid Monte Carlo?
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
42
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
43 3. USAGE PATTERNS
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
44 =================
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
45
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
46 * MCMC methods have a usage pattern that is quite different from the kind of univariate sampling methods
1189
0e12ea6ba661 fix many rst syntax error warning.
Frederic Bastien <nouiz@nouiz.org>
parents: 1040
diff changeset
47 needed for nice-and-easy parametric families.
1027
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
48
a1b6ccd5b6dc few comments added
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 1018
diff changeset
49