Mercurial > pylearn

annotate doc/v2_planning/plugin_JB.py @ 1198:1387771296a8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
v2planning adding plugin_JB
author James Bergstra <bergstrj@iro.umontreal.ca>
date Mon, 20 Sep 2010 02:34:23 -0400
parents
children 98954d8cb92d
rev   line source
1198
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1 """plugin_JB - draft of library architecture using iterators"""
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
3
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
4 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
5
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
6 - PICKLABLE - algorithms are serializable at all points during execution
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
7
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
8 - ITERATOR walks through algorithms with fine granularity
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
9
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
10 - COMPONENTS - library provides components on which programs operate
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
11
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
12 - ALGORITHMS - library provides algorithms in clean (no hooks) form
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
13
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
14 - HOOKS - user can insert print / debug logic with search/replace type calls
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
15 e.g. prog.find(CALL(cd1_update)).replace_with(SEQ([CALL(cd1_update), CALL(debugfn)]))
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
16
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
17 - PRINTING - user can print the 'program code' of an algorithm built from library pieces
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
18
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
19 - MODULAR EXPERIMENTS - an experiment object with one (or more?) programs and all of the objects referred to by
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
20 those programs. It is the preferred type of object to be serialized. The main components of
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
21 the algorithms should be top-level attributes of the package. This object can be serialized
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
22 and loaded in another process to implement job migration.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
23
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
24 - OPTIMIZATION - program can be optimized automatically
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
25 e.g. BUFFER(N, CALL(dataset.next)) can be replaced if dataset.next implements the right
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
26 attribute/protocol for 'bufferable' or something.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
27
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
28 e.g. SEQ([a,b,c,d]) can be compiled with Theano if sub-sequence is compatible
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
29
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
30 - don't need greenlets to get efficiency, the implementations of control flow ops can manage a
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
31 stack or stack tree in the vm (like greenlets do I think) we don't really need
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
32 greenlets/stackless I don't think
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
33
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
34 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
35
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
36 __license__ = None
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
37 __copyright__ = None
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
38
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
39 import copy, sys, cPickle
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
40
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
41 import numpy
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
42
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
43
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
44 ###################################################
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
45 # Virtual Machine for executing programs
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
46
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
47 class VirtualMachine(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
48 def __init__(self, prog):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
49 self.prog = prog
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
50 self.started = False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
51 self.finished=False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
52 def __iter__(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
53 assert not self.started
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
54 self.prog.start(None)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
55 self.started = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
56 return self
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
57 def next(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
58 if self.finished:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
59 raise StopIteration()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
60 r = self.prog.step()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
61 if r is INCOMPLETE:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
62 return r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
63 else:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
64 self.finished=True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
65 return r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
66 def run(self,n_steps=float('inf')):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
67 i = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
68 for r in self:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
69 i += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
70 if i > n_steps:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
71 break
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
72 return r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
73
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
74
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
75 ####################################################
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
76 # CONTROL-FLOW CONSTRUCTS
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
77
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
78 class INCOMPLETE:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
79 """Return value for Element.step"""
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
80
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
81 class ELEMENT(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
82 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
83 every execution block has a driver
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
84
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
85 the driver calls start when entering a new control element
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
86 - this would be called once per e.g. outer loop iteration
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
87
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
88 the driver calls step to advance the control element
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
89 - which returns INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
90 - which returns any other object to indicate completion
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
91 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
92
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
93 def start(self, arg):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
94 pass
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
95 def step(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
96 pass
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
97
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
98
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
99 class BUFFER_REPEAT(ELEMENT):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
100 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
101 Accumulate a number of return values into one list / array.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
102
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
103 The source of return values `src` is a control element that will be restarted repeatedly in
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
104 order to fulfil the requiement of gathering N samples.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
105
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
106 TODO: support accumulating of tuples of arrays
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
107 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
108 def __init__(self, N, src, storage=None):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
109 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
110 TODO: use preallocated `storage`
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
111 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
112 self.N = N
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
113 self.n = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
114 self.src = src
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
115 self.storage = storage
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
116 self.src.start(None)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
117 if self.storage != None:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
118 raise NotImplementedError()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
119 def start(self, arg):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
120 self.buf = [None] * self.N
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
121 self.n = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
122 self.finished = False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
123 def step(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
124 assert not self.finished
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
125 r = self.src.step()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
126 if r is INCOMPLETE:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
127 return r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
128 self.src.start(None) # restart our stream
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
129 self.buf[self.n] = r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
130 self.n += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
131 if self.n == self.N:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
132 self.finished = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
133 return self.buf
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
134 else:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
135 return INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
136 assert 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
137
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
138 class CALL(ELEMENT):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
139 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
140 Control flow terminal - call a python function or method.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
141
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
142 Returns the return value of the call.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
143 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
144 def __init__(self, fn, *args, **kwargs):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
145 self.fn = fn
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
146 self.args = args
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
147 self.kwargs=kwargs
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
148 self.use_start_arg = kwargs.pop('use_start_arg', False)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
149 def start(self, arg):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
150 self.start_arg = arg
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
151 self.finished = False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
152 return self
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
153 def step(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
154 assert not self.finished
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
155 self.finished = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
156 if self.use_start_arg:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
157 if self.args:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
158 raise TypeError('cant get positional args both ways')
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
159 return self.fn(self.start_arg, **self.kwargs)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
160 else:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
161 return self.fn(*self.args, **self.kwargs)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
162 def __getstate__(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
163 rval = self.__dict__
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
164 if type(self.fn) is type(self.step): #instancemethod
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
165 fn = rval.pop('fn')
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
166 rval['i fn'] = fn.im_func, fn.im_self, fn.im_class
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
167 return rval
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
168 def __setstate__(self, dct):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
169 if 'i fn' in dct:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
170 dct['fn'] = type(self.step)(*dct.pop('i fn'))
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
171 self.__dict__.update(dct)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
172
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
173 def FILT(*args, **kwargs):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
174 return CALL(use_start_arg=True, *args, **kwargs)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
175
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
176 def CHOOSE(which, options):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
177 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
178 Execute one out of a number of optional control flow paths
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
179 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
180 raise NotImplementedError()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
181
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
182 def LOOP(elements):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
183 #TODO: implement a true infinite loop
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
184 try:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
185 iter(elements)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
186 return REPEAT(sys.maxint, elements)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
187 except TypeError:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
188 return REPEAT(sys.maxint, [elements])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
189
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
190 class REPEAT(ELEMENT):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
191 def __init__(self, N, elements, pass_rvals=False):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
192 self.N = N
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
193 self.elements = elements
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
194 self.pass_rvals = pass_rvals
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
195 #TODO: check for N being callable
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
196 def start(self, arg):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
197 self.n = 0 #loop iteration
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
198 self.idx = 0 #element idx
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
199 self.finished = False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
200 self.elements[0].start(arg)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
201 def step(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
202 assert not self.finished
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
203 r = self.elements[self.idx].step()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
204 if r is INCOMPLETE:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
205 return INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
206 self.idx += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
207 if self.idx < len(self.elements):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
208 self.elements[self.idx].start(r)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
209 return INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
210 self.n += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
211 if self.n < self.N:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
212 self.idx = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
213 self.elements[self.idx].start(r)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
214 return INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
215 else:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
216 self.finished = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
217 return r
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
218
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
219 def SEQ(elements):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
220 return REPEAT(1, elements)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
221
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
222 class WEAVE(ELEMENT):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
223 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
224 Interleave execution of a number of elements.
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
225
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
226 TODO: allow a schedule (at least relative frequency) of elements from each program
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
227 """
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
228 def __init__(self, elements):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
229 self.elements = elements
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
230 def start(self, arg):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
231 for el in self.elements:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
232 el.start(arg)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
233 self.idx = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
234 self.any_is_finished = False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
235 self.finished= False
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
236 def step(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
237 assert not self.finished # if this is triggered, we have a broken driver
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
238 self.idx = self.idx % len(self.elements)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
239 r = self.elements[self.idx].step()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
240 if r is not INCOMPLETE:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
241 self.any_is_finished = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
242 self.idx += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
243 if self.idx == len(self.elements) and self.any_is_finished:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
244 self.finished = True
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
245 return None # dummy completion value
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
246 else:
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
247 return INCOMPLETE
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
248
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
249
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
250 ####################################################
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
251 # [Dummy] Components involved in learning algorithms
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
252
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
253 class Dataset(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
254 def __init__(self, data):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
255 self.pos = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
256 self.data = data
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
257 def next(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
258 rval = self.data[self.pos]
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
259 self.pos += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
260 if self.pos == len(self.data):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
261 self.pos = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
262 return rval
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
263 def seek(self, pos):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
264 self.pos = pos
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
265
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
266 class KFold(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
267 def __init__(self, data, K):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
268 self.data = data
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
269 self.k = -1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
270 self.scores = [None]*K
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
271 self.K = K
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
272 def next_fold(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
273 self.k += 1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
274 self.data.seek(0) # restart the stream
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
275 def next(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
276 #TODO: skip the examples that are ommitted in this split
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
277 return self.data.next()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
278 def init_test(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
279 pass
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
280 def next_test(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
281 return self.data.next()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
282 def test_size(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
283 return 5
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
284 def store_scores(self, scores):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
285 self.scores[self.k] = scores
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
286
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
287 class PCA_Analysis(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
288 def __init__(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
289 self.clear()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
290
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
291 def clear(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
292 self.mean = 0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
293 self.eigvecs=0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
294 self.eigvals=0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
295 def analyze(self, X):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
296 self.mean = numpy.mean(X, axis=0)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
297 self.eigvecs=1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
298 self.eigvals=1
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
299 def filt(self, X):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
300 return (X - self.mean) * self.eigvecs #TODO: divide by root eigvals?
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
301 def pseudo_inverse(self, Y):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
302 return Y
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
303
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
304 class Layer(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
305 def __init__(self, w):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
306 self.w = w
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
307 def filt(self, x):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
308 return self.w*x
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
309 def clear(self):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
310 self.w =0
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
311
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
312 def print_obj(obj):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
313 print obj
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
314 def print_obj_attr(obj, attr):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
315 print getattr(obj, attr)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
316 def no_op(*args, **kwargs):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
317 pass
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
318
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
319 class cd1_update(object):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
320 def __init__(self, layer, lr):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
321 self.layer = layer
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
322 self.lr = lr
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
323
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
324 def __call__(self, X):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
325 # update self.layer from observation X
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
326 self.layer.w += X.mean() * self.lr #TODO: not exactly correct math
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
327
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
328 def simple_main():
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
329
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
330 l = [0]
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
331 def f(a):
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
332 print l
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
333 l[0] += a
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
334 return l[0]
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
335
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
336 print VirtualMachine(WEAVE([
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
337 BUFFER_REPEAT(3,CALL(f,1)),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
338 BUFFER_REPEAT(5,CALL(f,1)),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
339 ])).run()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
340
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
341 def main():
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
342 # create components
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
343 dataset = Dataset(numpy.random.RandomState(123).randn(13,1))
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
344 pca = PCA_Analysis()
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
345 layer1 = Layer(w=4)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
346 layer2 = Layer(w=3)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
347 kf = KFold(dataset, K=10)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
348
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
349 # create algorithm
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
350
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
351 train_pca = SEQ([
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
352 BUFFER_REPEAT(1000, CALL(kf.next)),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
353 FILT(pca.analyze)])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
354
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
355 train_layer1 = REPEAT(10, [
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
356 BUFFER_REPEAT(10, CALL(kf.next)),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
357 FILT(pca.filt),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
358 FILT(cd1_update(layer1, lr=.01))])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
359
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
360 train_layer2 = REPEAT(10, [
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
361 BUFFER_REPEAT(10, CALL(kf.next)),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
362 FILT(pca.filt),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
363 FILT(layer1.filt),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
364 FILT(cd1_update(layer2, lr=.01))])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
365
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
366 train_prog = SEQ([
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
367 train_pca,
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
368 WEAVE([
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
369 train_layer1,
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
370 LOOP(CALL(print_obj_attr, layer1, 'w'))]),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
371 train_layer2,
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
372 ])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
373
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
374 kfold_prog = REPEAT(10, [
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
375 CALL(kf.next_fold),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
376 CALL(pca.clear),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
377 CALL(layer1.clear),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
378 CALL(layer2.clear),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
379 train_prog,
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
380 CALL(kf.init_test),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
381 BUFFER_REPEAT(kf.test_size(),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
382 SEQ([
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
383 CALL(kf.next_test),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
384 FILT(pca.filt), # may want to allow this SEQ to be
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
385 FILT(layer1.filt), # optimized into a shorter one that
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
386 FILT(layer2.filt),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
387 FILT(numpy.mean)])), # chains together theano graphs
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
388 FILT(kf.store_scores),
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
389 ])
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
390
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
391 vm = VirtualMachine(kfold_prog)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
392
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
393 #vm2 = copy.deepcopy(vm)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
394 vm.run(n_steps=200000)
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
395 print kf.scores
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
396
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
397
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
398 if __name__ == '__main__':
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
399 sys.exit(main())
1387771296a8 v2planning adding plugin_JB
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
400