Mercurial > pylearn

annotate doc/v2_planning/plugin_greenlet.py @ 1198:1387771296a8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
v2planning adding plugin_JB
author James Bergstra <bergstrj@iro.umontreal.ca>
date Mon, 20 Sep 2010 02:34:23 -0400
parents a60b3472c4ba
children acfd5e747a75
rev   line source
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
1 """plugin_greenlet - draft of library architecture using greenlets"""
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
2
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
3
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
4 """
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
5
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
6 - PICKLABLE - algorithms are serializable at all points during execution
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
7
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
8 - ITERATOR walks through algorithms with fine granularity
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
9
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
10 - COMPONENTS - library provides components on which programs operate
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
11
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
12 - ALGORITHMS - library provides algorithms in clean (no hooks) form
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
13
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
14 - HOOKS - user can insert print / debug logic with search/replace type calls
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
15 e.g. prog.find(CALL(cd1_update)).replace_with(SEQ([CALL(cd1_update), CALL(debugfn)]))
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
16
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
17 - PRINTING - user can print the 'program code' of an algorithm built from library pieces
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
18
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
19 - MODULAR EXPERIMENTS - an experiment object with one (or more?) programs and all of the objects referred to by
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
20 those programs. It is the preferred type of object to be serialized. The main components of
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
21 the algorithms should be top-level attributes of the package. This object can be serialized
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
22 and loaded in another process to implement job migration.
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
23
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
24 - OPTIMIZATION - program can be optimized automatically
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
25 e.g. BUFFER(N, CALL(dataset.next)) can be replaced if dataset.next implements the right
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
26 attribute/protocol for 'bufferable' or something.
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
27
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
28 e.g. SEQ([a,b,c,d]) can be compiled with Theano if sub-sequence is compatible
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
29
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
30 - don't need greenlets to get efficiency, the implementations of control flow ops can manage a
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
31 stack or stack tree in the vm (like greenlets do I think) we don't really need
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
32 greenlets/stackless I don't think
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
33
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
34 """
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
35
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
36 __license__ = None
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
37 __copyright__ = None
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
38
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
39 import copy, sys
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
40
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
41 import numpy
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
42 from greenlet import greenlet
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
43
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
44 def vm_unpack(incoming):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
45 # can't reliably distinguish between a kwargs-only switch and a switch with one dict
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
46 # argument
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
47 if incoming is None:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
48 rval = (), {}
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
49 if isinstance(incoming, dict):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
50 rval = (), incoming
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
51 elif isinstance(incoming, tuple):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
52 if (len(incoming)==2
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
53 and isinstance(incoming[0], tuple)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
54 and isinstance(incoming[1], dict)):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
55 rval = incoming
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
56 else:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
57 rval = incoming, {}
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
58 else:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
59 rval = (incoming,), {}
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
60 #print 'unpack', incoming, rval
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
61 return rval[0][0], rval[0][1], rval[0][2:], rval[1]
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
62
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
63 def unpack_from_vm(incoming):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
64 assert isinstance(incoming, tuple)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
65 assert len(incoming)==4
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
66 return incoming
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
67
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
68 def vm_run(prog, *args, **kwargs):
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
69 #TODO: make this into a class with different ways to start the loop
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
70 # for example, if the (gr, dest, a, kw) tuple is returned,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
71 # then a program could be run for N steps and then paused,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
72 # saved, and restarted.
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
73 n_steps = kwargs.pop('n_steps', float('inf'))
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
74 def vm_loop(gr, dest, a, kw):
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
75 loop_iter = 0
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
76 while loop_iter < n_steps:
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
77 if gr == 'return':
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
78 break
1196
e9bb3340a870 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1195
diff changeset
79 #print 'vm_loop gr=',gr,'args=',a, 'kwargs=', kw
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
80 gr, dest, a, kw = gr.switch(vm, gr, dest, a, kw)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
81 #print 'gmain incoming', incoming
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
82 loop_iter += 1
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
83 # permit restarting
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
84 return gr, dest, a, kw
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
85 vm = greenlet(vm_loop)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
86 return vm.switch(prog, 'return', args, kwargs)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
87
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
88 ####################################################
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
89 # CONTROL-FLOW CONSTRUCTS
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
90
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
91 def SEQ(glets):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
92 return REPEAT(1, glets)
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
93
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
94 def REPEAT(N, glets):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
95 def repeat_task(vm, gself, dest, args, kwargs):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
96 while True:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
97 for i in xrange(N):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
98 for glet in glets:
1196
e9bb3340a870 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1195
diff changeset
99 #print 'repeat_task_i dest=%(dest)s args=%(args)s, kw=%(kwargs)s'%locals()
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
100 # jump to task `glet`
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
101 # with instructions to report results back to this loop `g`
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
102 _vm, _gself, _dest, args, kwargs = vm.switch(glet, gself, args, kwargs)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
103 assert _gself is gself
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
104 assert _dest is None # instructions can't tell us where to jump
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
105 vm, gself, dest, args, kwargs = vm.switch(dest, None, args, kwargs)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
106 return greenlet(repeat_task)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
107
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
108 def LOOP(seq):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
109 #TODO: implement a true infinite loop
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
110 try:
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
111 iter(seq)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
112 return REPEAT(sys.maxint, seq)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
113 except TypeError:
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
114 return REPEAT(sys.maxint, [seq])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
115
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
116 def CHOOSE(which, options):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
117 raise NotImplementedError()
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
118
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
119 def WEAVE(threads):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
120 def weave_task(vm, gself, dest, args, kwargs):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
121 # weave works by telling its threads that *it* is the vm
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
122 # and reporting back to the real vm indirectly
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
123 while True: # execution of weave is an iteration through this loop
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
124 # initially broadcast the args and kwargs to all threads
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
125 all_threads_live = True
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
126 thread_info = [(t, 'return', args, kwargs) for t in threads]
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
127 #print 'weave start -------------'
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
128 while all_threads_live:
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
129 #print 'weave iter'
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
130 for i in xrange(len(threads)):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
131 t_next, t_dest, t_args, t_kwargs = thread_info[i]
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
132 #tell the vm we're up to something, but ask it to come right back
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
133 #print 'weave 1'
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
134 _ignore = vm.switch(gself, None, (), {})
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
135
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
136 # pretend we're the vm_loop and tell the
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
137 # thread to advance by one and report back to us
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
138 #print 'weave 2', thread_info[i]
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
139 thread_info[i] = t_next.switch(gself, t_next, t_dest, t_args, t_kwargs)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
140 #print 'weave 3', thread_info[i]
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
141 if thread_info[i][0] is 'return':
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
142 #print 'thread has finished', i
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
143 all_threads_live = False
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
144
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
145 # some thread has died so we return control to parent
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
146 #####print 'weave returning', dest, args, kwargs
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
147 vm, gself, dest, args, kwargs = vm.switch(dest, None, args, kwargs)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
148 #####print 'weave continuing', dest, args, kwargs
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
149 return greenlet(weave_task)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
150
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
151 def BUFFER(N, glet):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
152 def BUFFER_loop(vm, gself, dest, args, kwargs):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
153 while True: #body runs once per execution
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
154 buf = []
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
155 for i in xrange(N):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
156 # jump to task `glet`
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
157 # with instructions to report results back to this loop `g`
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
158 _vm, _gself, _dest, _args, _kwargs = vm.switch(glet, gself, args, kwargs)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
159 buf.append(_args[0])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
160 assert len(_args)==1
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
161 assert _kwargs=={}
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
162 assert _gself is gself
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
163 assert _dest is None # instructions can't tell us where to jump
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
164 buf = numpy.asarray(buf)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
165 vm, gself, dest, args, kwargs = vm.switch(dest, None, (buf,), {})
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
166 return greenlet(BUFFER_loop)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
167
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
168 def CALL(fn):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
169 """
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
170 Create a greenlet whose first argument is the return-jump location.
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
171
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
172 fn must accept as the first positional argument this greenlet itself, which can be used as
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
173 the return-jump location for internal greenlet switches (ideally using gswitch).
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
174 """
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
175 def CALL_loop(vm, gself, dest, args, kwargs):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
176 while True:
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
177 #print 'CALL calling', fn.__name__, args, kwargs
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
178 t = fn(*args, **kwargs)
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
179 #TODO consider a protocol for returning args, kwargs
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
180 if t is None:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
181 _vm,_gself,dest, args, kwargs = vm.switch(dest, None, (), {})
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
182 else:
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
183 _vm,_gself,dest, args, kwargs = vm.switch(dest, None, (t,), {})
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
184
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
185 assert gself is _gself
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
186 return greenlet(CALL_loop)
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
187
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
188 ####################################################
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
189 # Components involved in the learning process
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
190
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
191 class Dataset(object):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
192 def __init__(self, data):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
193 self.pos = 0
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
194 self.data = data
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
195 def next(self):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
196 rval = self.data[self.pos]
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
197 self.pos += 1
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
198 if self.pos == len(self.data):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
199 self.pos = 0
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
200 return rval
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
201 def seek(self, pos):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
202 self.pos = pos
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
203
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
204 class KFold(object):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
205 def __init__(self, data, K):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
206 self.data = data
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
207 self.k = -1
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
208 self.scores = [None]*K
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
209 self.K = K
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
210 def next_fold(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
211 self.k += 1
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
212 self.data.seek(0) # restart the stream
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
213 def next(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
214 #TODO: skip the examples that are ommitted in this split
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
215 return self.data.next()
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
216 def init_test(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
217 pass
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
218 def next_test(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
219 return self.data.next()
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
220 def test_size(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
221 return 5
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
222 def store_scores(self, scores):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
223 self.scores[self.k] = scores
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
224
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
225 class PCA_Analysis(object):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
226 def __init__(self):
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
227 self.clear()
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
228
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
229 def clear(self):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
230 self.mean = 0
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
231 self.eigvecs=0
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
232 self.eigvals=0
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
233 def analyze(self, X):
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
234 self.mean = X.mean(axis=0)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
235 self.eigvecs=1
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
236 self.eigvals=1
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
237 def filt(self, X):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
238 return (X - self.mean) * self.eigvecs #TODO: divide by root eigvals?
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
239 def pseudo_inverse(self, Y):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
240 return Y
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
241
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
242 class Layer(object):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
243 def __init__(self, w):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
244 self.w = w
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
245 def filt(self, x):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
246 return self.w*x
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
247 def clear(self):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
248 self.w =0
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
249
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
250 def print_obj(vm, gself, obj):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
251 print obj
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
252 def no_op(*args, **kwargs):
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
253 pass
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
254
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
255 class cd1_update(object):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
256 def __init__(self, layer, lr):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
257 self.layer = layer
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
258 self.lr = lr
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
259
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
260 def __call__(self, X):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
261 # update self.layer from observation X
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
262 print 'cd1', X
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
263 print X.mean()
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
264 self.layer.w += X.mean() * self.lr #TODO: not exactly correct math
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
265
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
266 def main():
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
267 # create components
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
268 dataset = Dataset(numpy.random.RandomState(123).randn(13,1))
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
269 pca = PCA_Analysis()
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
270 layer1 = Layer(w=4)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
271 layer2 = Layer(w=3)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
272 kf = KFold(dataset, K=10)
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
273
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
274 # create algorithm
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
275
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
276 train_pca = SEQ([
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
277 BUFFER(1000, CALL(kf.next)),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
278 CALL(pca.analyze)])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
279
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
280 train_layer1 = REPEAT(10, [
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
281 BUFFER(10, CALL(kf.next)),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
282 CALL(pca.filt),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
283 CALL(cd1_update(layer1, lr=.01))])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
284
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
285 train_layer2 = REPEAT(10, [
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
286 BUFFER(10, CALL(kf.next)),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
287 CALL(pca.filt),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
288 CALL(layer1.filt),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
289 CALL(cd1_update(layer2, lr=.01))])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
290
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
291 def print_layer_w(*a,**kw):
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
292 print layer1.w
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
293
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
294 train_prog = SEQ([
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
295 train_pca,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
296 WEAVE([
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
297 train_layer1,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
298 LOOP(CALL(print_layer_w))]),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
299 train_layer2,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
300 ])
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
301
1197
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
302 kfold_prog = REPEAT(10, [
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
303 CALL(kf.next_fold),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
304 CALL(pca.clear),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
305 CALL(layer1.clear),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
306 CALL(layer2.clear),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
307 train_prog,
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
308 CALL(kf.init_test),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
309 BUFFER(kf.test_size(),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
310 SEQ([
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
311 CALL(kf.next_test),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
312 CALL(pca.filt), # may want to allow this SEQ to be
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
313 CALL(layer1.filt), # optimized into a shorter one that
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
314 CALL(layer2.filt),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
315 CALL(numpy.mean)])), # chains together theano graphs
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
316 CALL(kf.store_scores),
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
317 ])
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
318
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
319 vm_run(kfold_prog, n_steps=500)
a60b3472c4ba more progress on greenlets
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1196
diff changeset
320 print kf.scores
1195
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
321
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
322
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
323 if __name__ == '__main__':
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
324 sys.exit(main())
d3ee0d2d03e6 plugin_greenlet draft0
James Bergstra <bergstrj@iro.umontreal.ca>
parents:
diff changeset
325