annotate doc/v2_planning/plugin_architecture_GD.txt @ 1139:9f0502f8c7a5

Example of the plugin architecture I had in mind
author gdesjardins
date Thu, 16 Sep 2010 13:27:17 -0400
parents
children 9ff2242a817b
rev   line source
1139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
1 Overview
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
2 ========
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
3
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
4 The "central authority" (CA) is the glue which takes care of interfacing plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
5 with one another. It has 3 basic roles:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
6 * it maintains a list of "registered" or "active" plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
7 * it receives and queues the various messages sent by the plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
8 * dispatches the messages to the recipient, based on various "events"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
9
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
10 Events can take different forms:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
11 * the CA can trigger various events based on running time
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
12 * can be linked to messages emitted by the various plugins. Events can be
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
13 triggered based on the frequency of such messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
14 * Once an event is triggered, it is relayed to the appropriate "recipient
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
15 plugin(s)"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
16
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
17 It is the responsibility of each plugin to inform the CA of which "events" it
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
18 cares about.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
19
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
20
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
21 Generic Pseudo-code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
22 ===================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
23
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
24 I'll try to write this in pseudo-python as best I can. I'll do this in
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
25 traditional OOP, as this is what I'm more comfortable with. I'll leave it up to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
26 James and OB to python-ize this :)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
27
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
28
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
29 class MessageX(Message):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
30 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
31 A message is basically a data container. This could very well be replaced by
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
32 a generic Python object.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
33 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
34
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
35 class Plugin(object):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
36 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
37 The base plugin object doesn't do much. It contains a reference to the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
38 (upon plugin being registered with the CA), provides boilerplate code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
39 for storing which "events" this plugin is susceptible to, as well as code
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
40 for registering callback functions for the various messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
41 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
42
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
43 CA = None # to be initialized upon plugin registration
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
44 active_msg = {} # dictionary of messages this plugin is susceptible to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
45 callbacks = {} # mapping of message class names --> callback function
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
46
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
47 def listen(msg_class, interval):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
48 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
49 :param msg_class: reference to the "message" class we are interested in.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
50 These messages will be forwarded to this plugin, when
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
51 the trigger condition is met.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
52 :param interval: integer. Forward the message to this plugin every 'interval'
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
53 such messages.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
54 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
55 self.active_msg[msg_class] = interval
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
56
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
57
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
58 def check_trigger(msg_class, time):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
59 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
60 Checks whether or not the "trigger" condition associated with message of
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
61 class 'msg_class' is satisfied or not. This could be the default
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
62 behavior, and be overridden by the various plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
63 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
64 return time % self.active_msg[msg_class] == 0
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
65
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
66
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
67 def handler(msg_class, callback):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
68 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
69 Decorator which registers a callback function for the given message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
70 type.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
71
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
72 NOTE: I don't think what I wrote would work as a Python decorator. I am
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
73 not sure how to handle decoraters with multiple parameters (one
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
74 explicit, and the other as the reference to the function). I'm pretty
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
75 sure James or OB could figure it out though !
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
76
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
77 :params msg_class: reference to the message class for which we are
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
78 registering a callback function
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
79 :params callback : reference to which function to call for a given message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
80 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
81
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
82 self.callbacks[msg_class] = callback
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
83
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
84
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
85 def execute(self, message):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
86 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
87 Boiler-plate code which executes the right callback function, for the
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
88 given message type.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
89 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
90 for (msg_class, callback) in self.callbacks.iteritems():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
91 if message.__class__ == msg_class:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
92 callback(message)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
93
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
94
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
95 class ProducerPlugin(Plugin):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
96
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
97 def dostuff():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
98 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
99 A typical "producer" plugin. It basically performs an arbitrary action
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
100 and asks the CA to forward the results (in the form of a message) to
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
101 other plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
102 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
103
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
104 # iteratively do stuff and relay messages to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
105 while(condition):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
106
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
107 msga = # do something
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
108 ca.send(msga) # ask CA to forward to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
109
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
110
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
111 class ConsumerPlugin(Plugin):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
112
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
113 @handler(MessageA)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
114 def func(msga):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
115 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
116 A consumer or "passive plugin" (eg. logger, etc). This function is
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
117 register as being the callback function for Message A objects.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
118 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
119 # do something with message A
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
120
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
121
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
122 class ConsumerProducerPlugin(Plugin):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
123
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
124 @handler(MessageA)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
125 def func(msga):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
126 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
127 Example of a consumer / producer plugin. It receives MessageA messages,
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
128 processes the data, then asks the CA to send a new message (MessageB) as
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
129 the result of its computation. The CA will automatically forward to all
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
130 interested parties.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
131
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
132 :param msga: MessageA instance
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
133 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
134
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
135 data = dostuff(msga) # process message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
136 msgb = MessageB(data) # generate new message for other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
137 ca.send(msgb) # ask CA to forward to other plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
138
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
139
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
140
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
141 class CentralAuthority(object):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
142
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
143 active_plugins = [] # contains a list of registered plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
144
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
145 mailmain = {} # dictionary which contains, for each message class, a
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
146 # list of plugins interested in this message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
147
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
148 event_count = {} # dictionary of "event" counts for various messages
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
149
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
150 def register(plugin):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
151 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
152 Registers the plugin and adds it as a listener for the various messages
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
153 it is interested in.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
154 :param plugin: plugin instance which we want to "activate"
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
155 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
156
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
157 # each plugin must have a reference to the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
158 plugin.ca = self
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
159
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
160 # maintain list of active plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
161 active_plugins.append(plugin)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
162
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
163 # remember which messages this plugin cares about
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
164 for msg in plugin.active_msg.keys():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
165 self.mailman[msg].append(plugin)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
166 self.event_count[msg] = 0
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
167
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
168 def send(msg):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
169 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
170 This function relays the message to the appropriate plugins, based on
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
171 their "trigger" condition. It also keeps track of the number of times
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
172 this event was raised.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
173
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
174 :param msg: message instance
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
175 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
176
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
177 event_count[msg.__class__] += 1
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
178
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
179 # for all plugins interested in this message ...
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
180 for plugin in self.mailman[msg.__class__]:
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
181
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
182 # check if trigger condition is met
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
183 if plugin.check_trigger(msg, self.event_count[msg.__class__]):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
184
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
185 # have the plugin execute the message
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
186 plugin.execute(msg)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
187
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
188
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
189 def run(self):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
190 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
191 This would be the main loop of the program. I won't go into details
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
192 because its still somewhat blurry in my head :) But basically, the CA
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
193 could be configured to send out its own messages, independently from all
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
194 other plugins.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
195
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
196 These could be "synchronous" messages such as: "5 seconds have passed",
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
197 or others such as "save state we are about to get killed".
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
198
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
199 NOTE: seems like this would almost have to live in its own thread ...
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
200 """
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
201
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
202 # the following would be parametrized obviously
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
203 while(True):
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
204 msg = ElapsedTimeMessage(5)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
205 self.send(msg)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
206 sleep(5)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
207
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
208
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
209
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
210 Putting it all-together
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
211 =======================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
212
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
213
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
214 def main():
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
215
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
216 ca = CentralAuthority()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
217
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
218 producer = ProducerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
219 ca.register(producer)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
220
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
221 consumer = ConsumerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
222 consumer.listen(MessageB, 1)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
223 ca.register(consumer))
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
224
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
225 other = ConsumerProducerPlugin()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
226 other.listen(MessageB, 10)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
227 ca.register(other)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
228
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
229 # this is the function call which gets the ball rolling
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
230 producer.dostuff()
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
231
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
232
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
233 DISCUSSION: blocking vs. non-blocking
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
234 =====================================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
235
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
236 In the above example, I used "blocking" sends. However it is not-clear that this
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
237 is the best option.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
238
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
239 In the example, the producer basically acts as the main loop. It relinquishes
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
240 control of the main loop when the CA decides to forward the message to other
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
241 plugins. Control will only be returned once the cascade of send/receives
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
242 initiated with MessageA is complete (all subplugins have processed MessageA and
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
243 any messages sent as a side-effect have also been processed).
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
244
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
245 This definitely imposes constraints on what the plugins can do, and how they do
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
246 it. For the type of single-processor / linear jobs we tend to run, this might be
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
247 enough (??).
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
248
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
249 The good news is that going forward, the above plugin architecture can also
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
250 scale to distributed systems, by changing the sends to be non-blocking. Plugins
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
251 could then live on different machines and process data as they see fit.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
252 Synchronization would be enforced by way of messages. In the above, the "main
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
253 producer" would thus become a consumer/producer who listens for "done processing
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
254 MessageA" messages and produces a new MessageA as a result.
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
255
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
256 On single-processor systems, the synchronization overhead might be too costly
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
257 however. That is something we would have to investigate. On the plus side
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
258 however, our plugins would be "future proof" and lend themselves well to the
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
259 type of "massively parallel jobs" we wish to run (i.e. meta-learners, etc.)
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
260
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
261
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
262
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
263 Logistic Regression
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
264 ===================
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
265
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
266
9f0502f8c7a5 Example of the plugin architecture I had in mind
gdesjardins
parents:
diff changeset
267 TO COME SOON (?)