annotate doc/v2_planning/plugin_architecture_GD.txt @ 1321:ebcb76b38817

tinyimages - added main script to whiten patches
author James Bergstra <bergstrj@iro.umontreal.ca>
date Sun, 10 Oct 2010 13:43:53 -0400
parents 9ff2242a817b
children
rev   line source
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
1 Overview
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
2 ========
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
3
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
4 The "central authority" (CA) is the glue which takes care of interfacing plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
5 with one another. It has 3 basic roles:
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
6
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
7 * it maintains a list of "registered" or "active" plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
8 * it receives and queues the various messages sent by the plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
9 * dispatches the messages to the recipient, based on various "events"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
10
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
11 Events can take different forms:
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
12
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
13 * the CA can trigger various events based on running time
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
14 * can be linked to messages emitted by the various plugins. Events can be
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
15 triggered based on the frequency of such messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
16 * Once an event is triggered, it is relayed to the appropriate "recipient
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
17 plugin(s)"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
18
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
19 It is the responsibility of each plugin to inform the CA of which "events" it
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
20 cares about.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
21
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
22
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
23 Generic Pseudo-code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
24 ===================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
25
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
26 I'll try to write this in pseudo-python as best I can. I'll do this in
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
27 traditional OOP, as this is what I'm more comfortable with. I'll leave it up to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
28 James and OB to python-ize this :)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
29
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
30
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
31 .. code-block:: python
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
32
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
33 class MessageX(Message):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
34 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
35 A message is basically a data container. This could very well be replaced by
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
36 a generic Python object.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
37 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
38
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
39 class Plugin(object):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
40 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
41 The base plugin object doesn't do much. It contains a reference to the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
42 (upon plugin being registered with the CA), provides boilerplate code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
43 for storing which "events" this plugin is susceptible to, as well as code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
44 for registering callback functions for the various messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
45 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
46
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
47 CA = None # to be initialized upon plugin registration
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
48 active_msg = {} # dictionary of messages this plugin is susceptible to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
49 callbacks = {} # mapping of message class names --> callback function
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
50
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
51 def listen(msg_class, interval):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
52 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
53 :param msg_class: reference to the "message" class we are interested in.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
54 These messages will be forwarded to this plugin, when
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
55 the trigger condition is met.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
56 :param interval: integer. Forward the message to this plugin every 'interval'
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
57 such messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
58 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
59 self.active_msg[msg_class] = interval
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
60
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
61
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
62 def check_trigger(msg_class, time):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
63 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
64 Checks whether or not the "trigger" condition associated with message of
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
65 class 'msg_class' is satisfied or not. This could be the default
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
66 behavior, and be overridden by the various plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
67 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
68 return time % self.active_msg[msg_class] == 0
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
69
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
70
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
71 def handler(msg_class, callback):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
72 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
73 Decorator which registers a callback function for the given message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
74 type.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
75
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
76 NOTE: I don't think what I wrote would work as a Python decorator. I am
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
77 not sure how to handle decoraters with multiple parameters (one
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
78 explicit, and the other as the reference to the function). I'm pretty
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
79 sure James or OB could figure it out though !
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
80
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
81 :params msg_class: reference to the message class for which we are
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
82 registering a callback function
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
83 :params callback : reference to which function to call for a given message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
84 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
85
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
86 self.callbacks[msg_class] = callback
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
87
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
88
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
89 def execute(self, message):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
90 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
91 Boiler-plate code which executes the right callback function, for the
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
92 given message type.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
93 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
94 for (msg_class, callback) in self.callbacks.iteritems():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
95 if message.__class__ == msg_class:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
96 callback(message)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
97
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
98
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
99 class ProducerPlugin(Plugin):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
100
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
101 def dostuff():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
102 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
103 A typical "producer" plugin. It basically performs an arbitrary action
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
104 and asks the CA to forward the results (in the form of a message) to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
105 other plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
106 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
107
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
108 # iteratively do stuff and relay messages to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
109 while(condition):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
110
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
111 msga = # do something
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
112 ca.send(msga) # ask CA to forward to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
113
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
114
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
115 class ConsumerPlugin(Plugin):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
116
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
117 @handler(MessageA)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
118 def func(msga):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
119 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
120 A consumer or "passive plugin" (eg. logger, etc). This function is
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
121 register as being the callback function for Message A objects.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
122 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
123 # do something with message A
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
124
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
125
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
126 class ConsumerProducerPlugin(Plugin):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
127
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
128 @handler(MessageA)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
129 def func(msga):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
130 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
131 Example of a consumer / producer plugin. It receives MessageA messages,
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
132 processes the data, then asks the CA to send a new message (MessageB) as
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
133 the result of its computation. The CA will automatically forward to all
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
134 interested parties.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
135
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
136 :param msga: MessageA instance
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
137 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
138
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
139 data = dostuff(msga) # process message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
140 msgb = MessageB(data) # generate new message for other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
141 ca.send(msgb) # ask CA to forward to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
142
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
143
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
144
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
145 class CentralAuthority(object):
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
146
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
147 active_plugins = [] # contains a list of registered plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
148
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
149 mailmain = {} # dictionary which contains, for each message class, a
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
150 # list of plugins interested in this message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
151
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
152 event_count = {} # dictionary of "event" counts for various messages
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
153
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
154 def register(plugin):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
155 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
156 Registers the plugin and adds it as a listener for the various messages
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
157 it is interested in.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
158 :param plugin: plugin instance which we want to "activate"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
159 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
160
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
161 # each plugin must have a reference to the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
162 plugin.ca = self
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
163
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
164 # maintain list of active plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
165 active_plugins.append(plugin)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
166
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
167 # remember which messages this plugin cares about
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
168 for msg in plugin.active_msg.keys():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
169 self.mailman[msg].append(plugin)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
170 self.event_count[msg] = 0
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
171
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
172 def send(msg):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
173 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
174 This function relays the message to the appropriate plugins, based on
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
175 their "trigger" condition. It also keeps track of the number of times
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
176 this event was raised.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
177
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
178 :param msg: message instance
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
179 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
180
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
181 event_count[msg.__class__] += 1
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
182
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
183 # for all plugins interested in this message ...
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
184 for plugin in self.mailman[msg.__class__]:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
185
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
186 # check if trigger condition is met
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
187 if plugin.check_trigger(msg, self.event_count[msg.__class__]):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
188
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
189 # have the plugin execute the message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
190 plugin.execute(msg)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
191
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
192
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
193 def run(self):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
194 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
195 This would be the main loop of the program. I won't go into details
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
196 because its still somewhat blurry in my head :) But basically, the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
197 could be configured to send out its own messages, independently from all
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
198 other plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
199
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
200 These could be "synchronous" messages such as: "5 seconds have passed",
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
201 or others such as "save state we are about to get killed".
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
202
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
203 NOTE: seems like this would almost have to live in its own thread ...
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
204 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
205
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
206 # the following would be parametrized obviously
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
207 while(True):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
208 msg = ElapsedTimeMessage(5)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
209 self.send(msg)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
210 sleep(5)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
211
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
212
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
213
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
214 Putting it all-together
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
215 =======================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
216
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
217
1190
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
218 .. code-block:: python
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
219
9ff2242a817b fix rst syntax errors/warnings
Frederic Bastien <nouiz@nouiz.org>
parents: 1139
diff changeset
220 def main():
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
221
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
222 ca = CentralAuthority()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
223
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
224 producer = ProducerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
225 ca.register(producer)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
226
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
227 consumer = ConsumerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
228 consumer.listen(MessageB, 1)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
229 ca.register(consumer))
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
230
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
231 other = ConsumerProducerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
232 other.listen(MessageB, 10)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
233 ca.register(other)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
234
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
235 # this is the function call which gets the ball rolling
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
236 producer.dostuff()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
237
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
238
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
239 DISCUSSION: blocking vs. non-blocking
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
240 =====================================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
241
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
242 In the above example, I used "blocking" sends. However it is not-clear that this
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
243 is the best option.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
244
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
245 In the example, the producer basically acts as the main loop. It relinquishes
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
246 control of the main loop when the CA decides to forward the message to other
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
247 plugins. Control will only be returned once the cascade of send/receives
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
248 initiated with MessageA is complete (all subplugins have processed MessageA and
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
249 any messages sent as a side-effect have also been processed).
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
250
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
251 This definitely imposes constraints on what the plugins can do, and how they do
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
252 it. For the type of single-processor / linear jobs we tend to run, this might be
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
253 enough (??).
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
254
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
255 The good news is that going forward, the above plugin architecture can also
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
256 scale to distributed systems, by changing the sends to be non-blocking. Plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
257 could then live on different machines and process data as they see fit.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
258 Synchronization would be enforced by way of messages. In the above, the "main
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
259 producer" would thus become a consumer/producer who listens for "done processing
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
260 MessageA" messages and produces a new MessageA as a result.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
261
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
262 On single-processor systems, the synchronization overhead might be too costly
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
263 however. That is something we would have to investigate. On the plus side
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
264 however, our plugins would be "future proof" and lend themselves well to the
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
265 type of "massively parallel jobs" we wish to run (i.e. meta-learners, etc.)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
266
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
267
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
268
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
269 Logistic Regression
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
270 ===================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
271
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
272
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
273 TO COME SOON (?)