annotate doc/v2_planning/sampler.txt @ 1018:790376d986a3

initial document for sampling
author gdesjardins
date Fri, 03 Sep 2010 15:01:02 -0400
parents
children a1b6ccd5b6dc
rev   line source
1018
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
1 OVERVIEW
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
2 ========
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
3
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
4 Before we start defining what a sampler is and how it should be defined in
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
5 pylearn, we should first know what we're up against.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
6
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
7 The workflow I have in mind is the following:
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
8 1. identify the most popular sampling algorithms in the litterature
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
9 2. get up to speed with methods we're not familiar with
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
10 3. identify common usage patterns, properties of the algorithm, etc.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
11 4. decide on an API / best way to implement them
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
12 5. prioritize the algorithms
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
13 6. code away
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
14
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
15 1.BACKGROUND
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
16 =============
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
17
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
18 This section should provide a brief overview of what exists in the litterature.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
19 We should make sure to have a decent understanding of all of these (not everyone
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
20 has to be experts though), so that we can *intelligently* design our sampler
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
21 interface based on common usage patterns, properties, etc.
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
22
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
23 Sampling from basic distributions
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
24 * already supported: uniform, normal, binomial, multinomial
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
25 * wish list: beta, poisson, others ?
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
26
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
27 List of sampling algorithms:
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
28
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
29 * inversion sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
30 * rejection sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
31 * importance sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
32 * Markov Chain Monte Carlo
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
33 * Gibbs sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
34 * Metropolis Hastings
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
35 * Slice Sampling
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
36 * Annealing
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
37 * Parallel Tempering, Tempered Transitions, Simulated Tempering
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
38 * Nested Sampling (?)
790376d986a3 initial document for sampling
gdesjardins
parents:
diff changeset
39 * Hamiltonian Monte Carlo