Mercurial > pylearn

annotate doc/v2_planning/API_learner.txt @ 1224:f68b857eb11b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
reply to comments on architecture document
author Ian Goodfellow
date Wed, 22 Sep 2010 15:44:14 -0400
parents 805e7c369fd1
children 317049b21b77
rev   line source
1175
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
1 .. _v2planning_learner:
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
2
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
3 Learner API
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
4 ===========
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
5
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
6 # A list of "task types"
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
7
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
8 '''
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
9 List of tasks types:
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
10 Attributes
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
11
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
12 sequential
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
13 spatial
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
14 structured
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
15 semi-supervised
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
16 missing-values
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
17
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
18
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
19 Supervised (x,y)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
20
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
21 classification
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
22 regression
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
23 probabilistic classification
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
24 ranking
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
25 conditional density estimation
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
26 collaborative filtering
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
27 ordinal regression ?= ranking
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
28
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
29 Unsupervised (x)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
30
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
31 de-noising
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
32 feature learning ( transformation ) PCA, DAA
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
33 density estimation
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
34 inference
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
35
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
36 Other
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
37
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
38 generation (sampling)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
39 structure learning ???
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
40
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
41
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
42 Notes on metrics & statistics:
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
43 - some are applied to an example, others on a batch
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
44 - most statistics are on the dataset
1175
805e7c369fd1 small change to fix warning and allow the file to generate a HTML page.
Frederic Bastien <nouiz@nouiz.org>
parents: 1168
diff changeset
45
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
46 '''
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
47
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
48
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
49 class Learner(Object):
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
50 '''
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
51 Takes data as inputs, and learns a prediction function (or several).
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
52
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
53 A learner is parametrized by hyper-parameters, which can be set from the
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
54 outside (a "client" from Learner, that can be a HyperLearner, a
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
55 Tester,...).
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
56
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
57 The data can be given all at a time as a data set, or incrementally.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
58 Some learner need to be fully trained in one step, whereas other can be
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
59 trained incrementally.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
60
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
61 The question of statistics collection during training remains open.
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
62 '''
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
63 #def use_dataset(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
64
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
65 # return a dictionary of hyperparameters names(keys)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
66 # and value(values)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
67 def get_hyper_parameters()
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
68 def set_hyper_parameters(dictionary)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
69
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
70
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
71
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
72
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
73 # Ver B
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
74 def eval(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
75 def predict(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
76
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
77 # Trainable
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
78 def train(dataset) # train until complition
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
79
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
80 # Incremental
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
81 def use_dataset(dataset)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
82 def adapt(n_steps =1)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
83 def has_converged()
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
84
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
85 #
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
86
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
87
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
88 # Some example cases
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
89
1087
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
90 class HyperLearner(Learner):
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
91
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
92 ### def get_hyper_parameter_distribution(name)
8c448829db30 learning committee first draft of an api
Razvan Pascanu <r.pascanu@gmail.com>
parents:
diff changeset
93 def set_hyper_parameters_distribution(dictionary)
1168
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
94
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
95
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
96 def bagging(learner_factory):
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
97 for i in range(N):
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
98 learner_i = learner_factory.new()
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
99 # todo: get dataset_i ??
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
100 learner_i.use_dataset(dataset_i)
77b6ed85d3f7 Update doc of learner's API
Pascal Lamblin <lamblinp@iro.umontreal.ca>
parents: 1167
diff changeset
101 learner_i.train()