Mercurial > pylearn
annotate doc/v2_planning/learner.txt @ 1030:a154c9b68239
dataset: Dumi confirmed as leader
author | Olivier Delalleau <delallea@iro> |
---|---|
date | Mon, 06 Sep 2010 20:45:45 -0400 |
parents | 38f799f8b6cd |
children | 88b296cfba50 |
rev | line source |
---|---|
1002
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
1 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
2 Discussion of Function Specification for Learner Types |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
3 ====================================================== |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
4 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
5 In its most abstract form, a learner is an object with the |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
6 following semantics: |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
7 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
8 * A learner has named hyper-parameters that control how it learns (these can be viewed |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
9 as options of the constructor, or might be set directly by a user) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
10 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
11 * A learner also has an internal state that depends on what it has learned. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
12 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
13 * A learner reads and produces data, so the definition of learner is |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
14 intimately linked to the definition of dataset (and task). |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
15 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
16 * A learner has one or more 'train' or 'adapt' functions by which |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
17 it is given a sample of data (typically either the whole training set, or |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
18 a mini-batch, which contains as a special case a single 'example'). Learners |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
19 interface with datasets in order to obtain data. These functions cause the |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
20 learner to change its internal state and take advantage to some extent |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
21 of the data provided. The 'train' function should take charge of |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
22 completely exploiting the dataset, as specified per the hyper-parameters, |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
23 so that it would typically be called only once. An 'adapt' function |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
24 is meant for learners that can operate in an 'online' setting where |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
25 data continually arrive and the control loop (when to stop) is to |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
26 be managed outside of it. For most intents and purposes, the |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
27 'train' function could also handle the 'online' case by providing |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
28 the controlled iterations over the dataset (which would then be |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
29 seen as a stream of examples). |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
30 * learner.train(dataset) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
31 * learner.adapt(data) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
32 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
33 * Different types of learners can then exploit their internal state |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
34 in order to perform various computations after training is completed, |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
35 or in the middle of training, e.g., |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
36 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
37 * y=learner.predict(x) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
38 for learners that see (x,y) pairs during training and predict y given x, |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
39 or for learners that see only x's and learn a transformation of it (i.e. feature extraction). |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
40 Here and below, x and y are tensor-like objects whose first index iterates |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
41 over particular examples in a batch or minibatch of examples. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
42 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
43 * p=learner.probability(examples) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
44 p=learner.log_probability(examples) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
45 for learners that can estimate probability density or probability functions, |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
46 note that example could be a pair (x,y) for learners that expect each example |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
47 to represent such a pair. The second form is provided in case the example |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
48 is high-dimensional and computations in the log-domain are numerically preferable. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
49 The first dimension of examples or of x and y is an index over a minibatch or a dataset. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
50 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
51 * p=learner.free_energy(x) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
52 for learners that can estimate a log unnormalized probability; the output has the same length as the input. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
53 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
54 * c=learner.costs(examples) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
55 returns a matrix of costs (one row per example, i.e., again the output has the same length |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
56 as the input), the first column of which represents the cost whose expectation |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
57 we wish to minimize over new samples from the unknown underlying data distribution. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
58 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
59 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
60 Some learners may be able to handle x's and y's that contain missing values. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
61 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
62 * For convenience, some of these operations could be bundled, e.g. |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
63 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
64 * [prediction,costs] = learner.predict_and_adapt((x,y)) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
65 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
66 * Some learners could include in their internal state not only what they |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
67 have learned but some information about recently seen examples that conditions |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
68 the expected distribution of upcoming examples. In that case, they might |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
69 be used, e.g. in an online setting as follows: |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
70 for (x,y) in data_stream: |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
71 [prediction,costs]=learner.predict((x,y)) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
72 accumulate_statistics(prediction,costs) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
73 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
74 * In some cases, each example is itself a (possibly variable-size) sequence |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
75 or other variable-size object (e.g. an image, or a video) |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
76 |
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
77 |
1026
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
78 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
79 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
80 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
81 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
82 |
1002
f82093bf4405
adding learner.txt and dataset.txt in v2_planning/
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff
changeset
|
83 |
1026
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
84 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
85 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
86 James's idea for Learner Interface |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
87 =================================== |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
88 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
89 Theory: |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
90 ------- |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
91 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
92 Think about the unfolding of a learning algorithm as exploring a path in a vast |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
93 directed graph. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
94 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
95 There are some source nodes, which are potential initial conditions for the |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
96 learning algorithm. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
97 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
98 At any node, there are a number of outgoing labeled edges that represent |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
99 distinct directions of exploration: like "allocate a model with N hidden units", |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
100 or "set the l1 weight decay on such-and-such units to 0.1" or "adapt for T |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
101 iterations" or "refresh the GPU dataset memory with the next batch of data". |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
102 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
103 Not all nodes have the same outgoing edge labels. The dataset, model, and |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
104 optimization algorithm implementations may each have their various |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
105 hyper-parameters with various restrictions on what values they can take, and |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
106 when they can be changed. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
107 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
108 Every move in this graph incurs some storage and computational expense, and |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
109 explores the graph. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
110 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
111 Learners typically engage in goal-directed exploration of this graph - for |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
112 example to find the node with the best validation-set performance given a |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
113 certain computational budget. We might often be interested in the best node |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
114 found. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
115 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
116 The predict(), log_probability(), free_energy() etc correspond to costs that we |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
117 can measure at any particular node (at some computational expense) to see how we |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
118 are doing in our exploration. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
119 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
120 Many semantically distinct components come into the definition of this graph: |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
121 the model (e.g. DAA) the dataset (e.g. an online one), the inference and |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
122 learning strategy. I'm not sure what to call this graph than an 'experiment |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
123 graph'... so I'll go with that for now. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
124 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
125 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
126 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
127 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
128 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
129 Use Cases |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
130 ---------- |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
131 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
132 Early stopping |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
133 ~~~~~~~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
134 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
135 Early stopping can be implemented as a learner that progresses along a |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
136 particular kind of edge (e.g. "train more") until a stopping criterion (in terms |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
137 of a cost computed from nodes along the path) is met. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
138 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
139 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
140 Grid Search |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
141 ~~~~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
142 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
143 Grid search is a learner policy that can be implemented in an experiment graph |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
144 where all paths have the form: |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
145 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
146 ( "set param 0 to X", |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
147 "set param 1 to Y", |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
148 ... , |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
149 "set param N to Z", |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
150 adapt, |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
151 [early stop...], |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
152 test) |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
153 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
154 It would explore all paths of this form and then return the best node. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
155 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
156 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
157 Stagewise learning of DBNs combined with early stopping and grid search |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
158 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
159 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
160 This would be a learner that is effective for experiment graphs that reflect the |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
161 greedy-stagewise optimization of DBNs. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
162 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
163 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
164 Boosting |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
165 ~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
166 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
167 Given an ExperimentGraph that permits re-weighting of examples, it is |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
168 straightforward to write a meta-ExperimentGraph around it that implements AdaBoost. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
169 A meta-meta-ExperimentGraph around that that does early-stopping would complete |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
170 the picture and make a useful boosting implementation. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
171 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
172 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
173 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
174 Implementation Details / API |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
175 ---------------------------- |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
176 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
177 ExperimentGraph |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
178 ~~~~~~~~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
179 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
180 One API that needs to be defined for this perspective to be practical is the |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
181 ExperimentGraph. I'll present it in terms of global functions, but an |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
182 object-oriented things probably makes more sense in the code itself. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
183 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
184 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
185 def explored_nodes(graph): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
186 """Return iterator over explored nodes (ints? objects?)""" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
187 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
188 def forget_nodes(graph, nodes): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
189 """Clear the nodes from memory (save space)""" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
190 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
191 def all_edges_from(graph, node): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
192 """Return iterator over all possible edges |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
193 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
194 Edges might be parametric - like "set learn_rate to (float)" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
195 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
196 Edges might contain a reference to their 'from' end... not sure. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
197 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
198 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
199 def explored_edges_from(graph, node): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
200 """Return the edges that have been explored |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
201 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
202 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
203 def add_node(graph, new_node): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
204 """add a node. It may be serialized.""" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
205 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
206 def add_edge(graph, edge): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
207 """add edge, it may be serialize""" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
208 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
209 def connect(graph, from_node, to_node, edge): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
210 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
211 to_node = None for un-explored edge |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
212 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
213 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
214 It makes sense to have one ExperimentGraph implementation for each storage |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
215 mechanism - Memory, JobMan, sqlite, couchdb, mongodb, etc. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
216 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
217 The nodes should be serializable objects (like the 'learner' objects in Yoshua's |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
218 text above, so that you can do node.learner.predict() if the edge leading to |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
219 `node` trained something new). |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
220 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
221 The nodes could also contain the various costs (train, valid, test), and other |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
222 experiment statistics that are node-specific. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
223 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
224 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
225 Some implementations might also include functions for asynchronous updating of |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
226 the ExperimentGraph: |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
227 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
228 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
229 ExperimentGraphEdge |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
230 ~~~~~~~~~~~~~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
231 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
232 The ExperimentGraph is primarily a dictionary container for nodes and edges. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
233 An ExperimentGraphEdge implementation is the model-dependent component that |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
234 actually interprets the edges as computations. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
235 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
236 def estimate_compute_time(graph, node, edge): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
237 """Return an estimated walltime expense for the computation""" |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
238 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
239 def compute_edge(graph, node, edge, async=False, priority=1): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
240 """Run the computations assocated with this graph edge, and store the |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
241 resulting 'to_node' to the graph when complete. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
242 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
243 If async is True, the function doesn't return until the graph is updated |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
244 with `to_node`. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
245 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
246 The priority is used by implementations that use cluster software or |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
247 something to manage a worker pool that computes highest-priority edges |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
248 first. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
249 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
250 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
251 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
252 def list_compute_queue(graph): |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
253 """Return edges scheduled for exploration (and maybe a handle for |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
254 where/when they started running and other backend details) |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
255 """ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
256 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
257 Different implementations of ExperimentGraphExplorer will correspond to |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
258 different experiments. There can also be ExperimentGraphExplorer |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
259 implementations that are proxies, and perform the computations in different |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
260 threads, or across ssh, or cluster software. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
261 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
262 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
263 Learner |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
264 ~~~~~~~ |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
265 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
266 A learner is a program that implements a policy for graph exploration by |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
267 exploiting the ExperimentGraph and ExperimentGraphEdge interfaces. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
268 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
269 The convenience of the API hinges on the extent to which we can implement |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
270 policies that work on different experiment-graphs (where the labels on the edges |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
271 and semantics are different). The use-cases above make me optimistic that it |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
272 will work sufficiently well to be worth doing in the absence of better ideas. |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
273 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
274 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
275 |
38f799f8b6cd
v2_planning - thoughts on learner
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
1002
diff
changeset
|
276 |