Out Of The Box

A new improvement in PHoneyC DOM emulation code was committed in SVN r1624. The idea is to better emulate the DOM behaviour depending on the selected browser personality. Let’s take a look at the code starting from the personalities definition in config.py.

39 UserAgents = [
40     (1,
41      “Internet Explorer 6.0 (Windows 2000)”,
42      “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
43      “Mozilla”,
44      “Microsoft Internet Explorer”,
45      “4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
46      “ie60″,
47     ),
48     (2,
49      “Internet Explorer 6.1 (Windows XP)”,
50      “Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
51      “Mozilla”,
52      “Microsoft Internet Explorer”,
53      “4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
54      “ie61″,
55     ),
56     (3,
57      “Internet Explorer 7.0 (Windows XP)”,
58      “Mozilla/4.0 (Windows; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)”,
59      “Mozilla”,
60      “Microsoft Internet Explorer”,
61      “4.0 (Windows; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)”,
62      “ie70″,
63     ),
64     (4,
65      “Internet Explorer 8.0 (Windows XP)”,
66      “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 1.5); .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
67      “Mozilla”,
68      “Microsoft Internet Explorer”,
69      “4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; (R1 1.5); .NET CLR 1.1.4322; .NET CLR 2.0.50727)”,
70      “ie80″,
71     ),
72 ]

It’s important to realize that each personality was added a tag (i.e. ie80). Taking a look at DOM/Window.py the following code can be seen.

229     def __init_methods(self):
230         for attr in dir(self):
231             prefix = “_Window__window_%s_” % (config.browserTag, )
232             if attr.startswith(prefix):
233                 p = attr.split(prefix)[1]
234                 self.__dict__[‘__cx’].add_global(p, getattr(self, attr))
235                 self.__dict__[‘__cx’].execute(“window.%s = %s;” % (p, p, ))

Let’s consider an example and assume the Internet Explorer 8.0 personality was selected. It’s easy to realize that the prefix would assume the value _Window__window_ie80_. A few simple wrappers were created, one per personality, to each method as shown in the following code.

340     def __window_back(self):
341         “””
342         Returns the window to the previous item in the history.
343         Syntax
344
345         window.back()
346
347         Parameters
348
349         None.
350         “””
351         pass
352
353     def __window_ie60_back(self):
354         self.__window_back()
355
356     def __window_ie61_back(self):
357         self.__window_back()
358
359     def __window_ie70_back(self):
360         self.__window_back()
361
362     def __window_ie80_back(self):
363         self.__window_back()
364
365     def __window_firefox_back(self):
366         self.__window_back()

This is a quite simple situation but what if you want to define addEventListener method just for Firefox personalities and attachEvent just for Internet Explorer ones? Really simple to do!

1191     def __window_attachEvent(self, sEvent, fpNotify):
1192         if dataetc.isevent(sEvent, ‘window’):
1193             self.__dict__[sEvent] = fpNotify
1194
1195     def __window_ie60_attachEvent(self, sEvent, fpNotify):
1196         self.__window_attachEvent(sEvent, fpNotify)
1197
1198     def __window_ie61_attachEvent(self, sEvent, fpNotify):
1199         self.__window_attachEvent(sEvent, fpNotify)
1200
1201     def __window_ie70_attachEvent(self, sEvent, fpNotify):
1202         self.__window_attachEvent(sEvent, fpNotify)
1203
1204     def __window_ie80_attachEvent(self, sEvent, fpNotify):
1205         self.__window_attachEvent(sEvent, fpNotify)
1206
1207
1208     def __window_detachEvent(self, sEvent, fpNotify):
1209         if sEvent in self.__dict__ and self.__dict__[sEvent] == fpNotify:
1210             del self.__dict__[sEvent]
1211
1212     def __window_ie60_detachEvent(self, sEvent, fpNotify):
1213         self.__window_detachEvent(sEvent, fpNotify)
1214
1215     def __window_ie61_detachEvent(self, sEvent, fpNotify):
1216         self.__window_detachEvent(sEvent, fpNotify)
1217
1218     def __window_ie70_detachEvent(self, sEvent, fpNotify):
1219         self.__window_detachEvent(sEvent, fpNotify)
1220
1221     def __window_ie80_detachEvent(self, sEvent, fpNotify):
1222         self.__window_detachEvent(sEvent, fpNotify)
1223
1224
1225     def __window_addEventListener(self, type, listener, useCapture = False):
1226         if dataetc.isevent(type, ‘window’):
1227             self.__dict__[type] = listener
1228
1229     def __window_firefox_addEventListener(self, type, listener, useCapture = False):
1230         self.__window_addEventListener(type, listener, useCapture = False)
1231
1232
1233     def __window_removeEventListener(self, type, listener, useCapture = False):
1234         if type in self.__dict__ and self.__dict__[type] == listener:
1235             del self.__dict__[type]
1236
1237     def __window_firefox_removeEventListener(self, type, listener, useCapture = False):
1238         self.__window_removeEventListener(type, listener, useCapture = False)

Moreover this approach could allow to insert specific code within the wrappers if needed while implementing the method functionalities in the higher level __window_<method_name> wrapper.

A few weeks ago I started reviewing the PHoneyC DOM emulation code and realized it was turning to be hard to maintain and debug due to a huge amount of undocumented (and sometimes awful) hacks. For this reason I decided it was time to patch (and sometimes rewrite from scratch) such code. These posts will describe how the new DOM emulation code will work. The patch is not available right now since I’m testing the code but plans exists to commit it in the PHoneyC SVN in the next days.

In this first post we will take a look at the Window object in DOM/Window.py. During object inizialization, the following code is executed.

156     def __init_context(self):
157         “””
158             Spidermonkey Context initialization.
159         “””
160         document = Document(self)
161         self.__dict__[‘__cx’] = self.__dict__[‘__rt’].new_context(alertlist = [])
162         self.__dict__[‘__sl’] = []
163         self.__dict__[‘__fl’] = [document]
164
165         self.__init_properties(document)
166         self.__init_methods()
167         self.__finalize_context()

Let’s go into further details. First of all Window object properties are initialized through the __init_properties method.

181     def __init_properties(self, document):
182         self.__dict__[‘__cx’].add_global(‘window’, self)
183         self.__dict__[‘__cx’].add_global(‘self’  , self)
184         self.__dict__[‘__cx’].execute(“window.window = window;”)
185
186         self.__dict__[‘__cx’].add_global(‘document’, document)
187         self.__dict__[‘__cx’].execute(“window.document = document;”)
188
189         self.__dict__[‘__cx’].add_global(‘location’, document.location)
190         self.__dict__[‘__cx’].execute(“window.location = location;”)
191
192         self.__dict__[‘__cx’].add_global(“ActiveXObject”, ActiveXObject)
193
194         self.__dict__[‘__cx’].add_global(“navigator”, Navigator())
195         self.__dict__[‘__cx’].execute(“window.navigator = navigator;”)
196
197         self.__dict__[‘__cx’].add_global(“screen”, unknown())
198         self.__dict__[‘__cx’].execute(“window.screen = screen;”)
199
200         if ‘top_window’ in self.__dict__[‘__root’].__dict__:
201             if self.__dict__[‘__referrer’]:
202                 top = self.__dict__[‘__referrer’]
203             else:
204                 top = self.__dict__[‘__root’].top_window
205         else:
206             top = self
207
208         self.__dict__[‘__cx’].add_global(“top”, top)
209         self.__dict__[‘__cx’].execute(“window.top = top;”)
210
211         self.__dict__[‘__cx’].add_global(“parent”, top)
212         self.__dict__[‘__cx’].execute(“window.parent = parent;”)
213
214         self.__dict__[‘__cx’].add_global(“history”, History(document))
215         self.__dict__[‘__cx’].execute(“window.history = history;”)
216
217         self.__dict__[‘__cx’].execute(“window.innerWidth = 400;”)
218         self.__dict__[‘__cx’].execute(“window.innerHeight = 200;”)
219
220         self.__init_undefined_properties()
221
222     def __init_undefined_properties(self):
223         properties = (‘external’, ‘opera’, )
224
225         for prop in properties:
226             self.__dict__[‘__cx’].execute(“window.%s = undefined;” % (prop, ))

The code should be straightforward to understand. The idea beyond it is really simple. Simply stated this code allows Python objects’ variables and methods to be accessible from JS. Let’s move to most interesting stuff. Following the __init_methods method is called.

228     def __init_methods(self):
229         for attr in dir(self):
230             if attr.startswith(‘_Window__window’):
231                 p = attr.split(‘_Window__window_’)[1]
232                 self.__dict__[‘__cx’].add_global(p, getattr(self, attr))
233                 self.__dict__[‘__cx’].execute(“window.%s = %s;” % (p, p, ))

Not so easy to understand? Let’s take a look to the definition of a method.

322     def __window_back(self):
323         “””
324         Returns the window to the previous item in the history.
325         Syntax
326
327         window.back()
328
329         Parameters
330
331         None.
332         “””
333         pass

This is a private class method since its name starts with __. “If you try to call a private method, Python will raise a slightly misleading exception, saying that the method does not exist. Of course it does exist, but it’s private, so it’s not accessible outside the class. Strictly speaking, private methods are accessible outside their class, just not easily accessible. Nothing in Python is truly private; internally, the names of private methods and attributes are mangled and unmangled on the fly to make them seem inaccessible by their given names.” (taken from Dive Into Python). We can access the __window_back method of the Window class by the name _Window__window_back. This is the black magic __init_methods use for initializing methods. It’s quite easy to realize that adding a new method is really easy. All you need is to simply define a method named __window_<window_method_name> and match the signature of such method. How to emulate such method it’s up to you but a simple pass could do the trick.

The last step happens in __finalize_context method.

169     def __finalize_context(self):
170         self.__dict__[‘__cx’].execute(“Event = function(){}”)
171         self.__dict__[‘__cx’].execute(“function CollectGarbage() {};”)
172         self.__dict__[‘__cx’].execute(“function quit() {};”)
173         self.__dict__[‘__cx’].execute(“function prompt() {};”)
174
175         for clsname in dataetc.classlist:
176             inits = {‘window’ : self,
177                      ‘tagName': dataetc.classtotag(clsname),
178                      ‘parser’ : None}
179             self.__dict__[‘__cx’].add_global(clsname, DOMObjectFactory(clsname, inits))

The most interesting code is in lines 175-179. First of all let’s take a look at the DOMObjectFactory code (DOM/ClassFactory.py) which is a genuine Python hack.

3 class DynamicDOMObject(DOMObject):
4     def __init__(self):
5         self.__dict__.update(self.inits)
6         DOMObject.__init__(self, self.window, self.tagName, self.parser)
7
8 def DOMObjectFactory(name, initializers):
9     return type(name, (DynamicDOMObject,), {‘inits’ : initializers})

After reading Python documentation it should be easy to understand how this code works and how it’s able to dynamically add new DOM objects to the context.

type(name, bases, dict)

Return a new type object. This is essentially a dynamic form of the class statement. The name string is the class name and becomes the __name__ attribute; the bases tuple itemizes the base classes and becomes the __bases__ attribute; and the dict dictionary is the namespace containing definitions for class body and becomes the __dict__ attribute. For example, the following two statements create identical type objects:

>>> class X(object):
… a = 1
…
>>> X = type(‘X’, (object,), dict(a=1))

What about the Window event handlers? They are handled with a different mechanism which can be fully understood just by analyzing how the new DOM emulation code preparses the pages deferring code execution until the last possible moment. I’ll analyze such feature in a future post in greater detail. Right now what you have to know is that if the handler for the event <event> is set, the Window attribute on<event> is set and contains the handler code. Once you understand it, the following code in DOM/DOM.py used for event handling should be easy to understand.

171     def get_event_func(self, name, f):
172         begin = str(f).index(‘{‘) + 1
173         s = str(f)[begin:].split(‘}’)
174         script = ‘}’.join(s[:-1]) + s[-1]
175         return script
176
177     def event_handler(self, window, name, f):
178         if name in window.__dict__:
179             try:
180                 script = self.get_event_func(name, f)
181                 window.__dict__[‘__cx’].execute(script)
182             except:
183                 #print str(f)
184                 traceback.print_exc()
185                 pass
186
187     def handle_events(self, window):
188         window.__dict__[‘__warning’] = False
189         self.event_handler(window, ‘onabort’         , window.onabort)
190         self.event_handler(window, ‘onbeforeunload’  , window.onbeforeunload)
191         self.event_handler(window, ‘onblur’          , window.onblur)
192         self.event_handler(window, ‘onchange’        , window.onchange)
193         self.event_handler(window, ‘onclick’         , window.onclick)
194         self.event_handler(window, ‘onclose’         , window.onclose)
195         self.event_handler(window, ‘oncontextmenu’   , window.oncontextmenu)
196         self.event_handler(window, ‘ondragdrop’      , window.ondragdrop)
197         self.event_handler(window, ‘onerror’         , window.onerror)
198         self.event_handler(window, ‘onfocus’         , window.onfocus)
199         self.event_handler(window, ‘onhashchange’    , window.hashchange)
200         self.event_handler(window, ‘onkeydown’       , window.onkeydown)
201         self.event_handler(window, ‘onkeypress’      , window.onkeypress)
202         self.event_handler(window, ‘onkeyup’         , window.onkeyup)
203         self.event_handler(window, ‘onload’          , window.onload)
204         self.event_handler(window, ‘onmousedown’     , window.onmousedown)
205         self.event_handler(window, ‘onmousemove’     , window.onmousemove)
206         self.event_handler(window, ‘onmouseout’      , window.onmouseout)
207         self.event_handler(window, ‘onmouseover’     , window.onmouseover)
208         self.event_handler(window, ‘onmouseup’       , window.onmouseup)
209         self.event_handler(window, ‘onmozorientation’, window.onmozorientation)
210         self.event_handler(window, ‘onpaint’         , window.onpaint)
211         self.event_handler(window, ‘onpopstate’      , window.onpopstate)
212         self.event_handler(window, ‘onreset’         , window.onreset)
213         self.event_handler(window, ‘onresize’        , window.onresize)
214         self.event_handler(window, ‘onscroll’        , window.onscroll)
215         self.event_handler(window, ‘onselect’        , window.onselect)
216         self.event_handler(window, ‘onsubmit’        , window.onsubmit)
217         self.event_handler(window, ‘onunload’        , window.onunload)
218         self.event_handler(window, ‘onpageshow’      , window.onpageshow)
219         self.event_handler(window, ‘onpagehide’      , window.onpagehide)
220         window.__dict__[‘__warning’] = True