Mercurial > pylearn

annotate doc/v2_planning/API_learner.txt @ 1229:515033d4d3bf

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
a first draft of layer committee
author Razvan Pascanu <r.pascanu@gmail.com>
date Wed, 22 Sep 2010 19:43:24 -0400
parents 805e7c369fd1
children 317049b21b77
rev   line source
1175
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
1 .. _v2planning_learner:
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
2
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
3 Learner API
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
4 ===========
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
5
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
6 # A list of "task types"
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
7
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
8 '''
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
9 List of tasks types:
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
10 Attributes
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
11
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
12 sequential
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
13 spatial
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
14 structured
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
15 semi-supervised
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
16 missing-values
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
17
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
18
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
19 Supervised (x,y)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
20
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
21 classification
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
22 regression
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
23 probabilistic classification
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
24 ranking
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
25 conditional density estimation
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
26 collaborative filtering
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
27 ordinal regression ?= ranking
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
28
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
29 Unsupervised (x)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
30
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
31 de-noising
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
32 feature learning ( transformation ) PCA, DAA
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
33 density estimation
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
34 inference
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
35
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
36 Other
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
37
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
38 generation (sampling)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
39 structure learning ???
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
40
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
41
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
42 Notes on metrics & statistics:
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
43 - some are applied to an example, others on a batch
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
44 - most statistics are on the dataset
1175
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
45
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
46 '''
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
47
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
48
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
49 class Learner(Object):
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
50 '''
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
51 Takes data as inputs, and learns a prediction function (or several).
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
52
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
53 A learner is parametrized by hyper-parameters, which can be set from the
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
54 outside (a "client" from Learner, that can be a HyperLearner, a
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
55 Tester,...).
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
56
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
57 The data can be given all at a time as a data set, or incrementally.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
58 Some learner need to be fully trained in one step, whereas other can be
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
59 trained incrementally.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
60
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
61 The question of statistics collection during training remains open.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
62 '''
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
63 #def use_dataset(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
64
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
65 # return a dictionary of hyperparameters names(keys)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
66 # and value(values)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
67 def get_hyper_parameters()
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
68 def set_hyper_parameters(dictionary)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
69
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
70
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
71
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
72
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
73 # Ver B
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
74 def eval(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
75 def predict(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
76
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
77 # Trainable
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
78 def train(dataset) # train until complition
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
79
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
80 # Incremental
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
81 def use_dataset(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
82 def adapt(n_steps =1)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
83 def has_converged()
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
84
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
85 #
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
86
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
87
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
88 # Some example cases
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
89
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
90 class HyperLearner(Learner):
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
91
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
92 ### def get_hyper_parameter_distribution(name)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
93 def set_hyper_parameters_distribution(dictionary)
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
94
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
95
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
96 def bagging(learner_factory):
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
97 for i in range(N):
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
98 learner_i = learner_factory.new()
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
99 # todo: get dataset_i ??
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
100 learner_i.use_dataset(dataset_i)
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
101 learner_i.train()