Tuesday, March 11, 2008

Javascript Source View II - Firebug vs Venkman

OK, last time, I was very happy to find out that venkman was using the XUL tree to represent the javascript source, while firebug was using html <a> tag and <span> tags.

As I was trying to find the script to load into the source view, I started to wonder how did those two software did it. I would imagine that we need to implement something that will
  1. load the file into a stream
  2. process the stream line by line

The findings from venkman actually amazed me once again, and along the way, I picked up a few other nice things ;-) So here it goes...

First, I was quite confident that I want to do it in xul tree way instead of the html way, so I started to scratch my head and buried my head into the venkman source code once again ;-) But before I went too far, I discovered that there are actually two different source views in venkman! One is called source, another one is called source2. source2 is the one that enables us to set breakpoints, and source doesn't do much, and it is hidden by default.
From venkman-views.xul
145
<!-- source2 view -->
146
147 <floatingview id="source2" title="&Source.label;" flex="1">
148 <vbox flex="1">
149 <tabbox id="source2-tabbox" flex="1">
150 <tabs id="source2-tabs">
151 <!--
152 We've got to put this placeholder tab in the tabs element to avoid
153 bogus strict warnings and exceptions.
154 -->
155 <tab id="source2-bloke" hidden="true"/>
156 </tabs>
157 <tabpanels id="source2-deck" flex="1"/>
158 </tabbox>
159 <textbox class="plain" id="source2-heading" readonly="true"/>
160 </vbox>
161 </floatingview>
162
163 <!-- source view -->
164
165 <floatingview id="source" title="&Source.label;" flex="1">
166 <vbox id="source-view-content" flex="1">
167 <toolbox>
168 <toolbar id="source-header" grippytooltiptext="&SourceHeader.tip;">
169 <label id="source-url" flex="1" crop="end"/>
170 </toolbar>
171 </toolbox>
172 <tree id="source-tree" flex="1" persist="width"
173 class="focusring"
174 onclick="console.views.source.onClick(event);"
175 onselect="console.views.source.onSelect(event);"
176 context="context:source">
177
178 <treecols>
179 <treecol id="source:col-0" width="20px"
180 display="&SourceCol0.display;" persist="hidden width"/>
181 <splitter class="tree-splitter"/>
182 <treecol id="source:col-1" width="50px"
183 display="&SourceCol1.display;" persist="hidden width"/>
184 <splitter class="tree-splitter"/>
185 <treecol id="source:col-2" flex="1" display=""
186 ignoreincolumnpicker="true" persist="hidden width"/>
187 </treecols>
188
189 <treechildren id="source-tree-body"/>
190
191 </tree>
192 </vbox>
193 </floatingview>
From venkman-views.js
2226 /*******************************************************************************
2227 * Source2 View
2228 *******************************************************************************/
2229
2230 console.views.source2 = new Object();
2231
2232 const VIEW_SOURCE2 = "source2";
2233 console.views.source2.viewId = VIEW_SOURCE2;
2234
2235 console.views.source2.init =
2236 function ss_init ()
2237 {

3527 /*******************************************************************************
3528 * Source View
3529 *******************************************************************************/
3530 console.views.source = new BasicOView();
3531
3532 const VIEW_SOURCE = "source";
3533 console.views.source.viewId = VIEW_SOURCE;
3534
3535 console.views.source.details = null;
3536 console.views.source.prettyPrint = false;
3537
3538 console.views.source.init =
3539 function sv_init()
3540 {

Notice that Source2 is an Object, while Source is a BasicOView. As what I usually do, I first scan the code for possible candidates of the key methods that I am looking for, and these two methods caught my attention.
3040 console.views.source2.loadSourceTextAtIndex =
3041 function s2v_loadsource (sourceText, index)
3042 {
3043 var sourceTab;
3044
3045 if (index in this.sourceTabList)
3046 {
3047 sourceTab = this.sourceTabList[index];
3048 sourceTab.content = null;
3049 sourceTab.sourceText = sourceText;
3050
3051 if ("currentContent" in this)
3052 {
3053 if (!ASSERT(sourceTab.tab,
3054 "existing sourcetab not fully initialized"))
3055 {
3056 return null;
3057 }
3058 sourceTab.tab.label.setAttribute("value", sourceText.shortName);
3059 sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
3060 sourceTab.iframe.setAttribute("raiseOnSync", "true");
3061 this.syncOutputFrame(sourceTab.iframe);
3062 }
3063
3064 }
3065 else
3066 {
3067 sourceTab = {
3068 sourceText: sourceText,
3069 tab: null,
3070 iframe: null,
3071 content: null
3072 };
3073
3074 this.sourceTabList[index] = sourceTab;
3075
3076 if ("currentContent" in this)
3077 {
3078 this.createFrameFor(sourceTab, index, true);
3079 }
3080 }
3081
3082 return sourceTab;
3083 }
3085 console.views.source2.addSourceText =
3086 function s2v_addsource (sourceText)
3087 {

addSourceText calls loadSourceTextAtIndex to load a specific javascript source into source2's tabbed source views. addSourceText function will just happily take the sourceText, and stash it into a tab ;-) Great, the sourceText explains it all! (Grrr...) So what the heck is sourceText? For a while, I thought it was real string, but I quickly had an impression that I am wrong, because strings are not smart enough to be displayed into a xultree. So who calls the addSourceText? I dig on...
3456 console.views.source2.hooks["hook-display-sourcetext"] =
3457 console.views.source2.hooks["hook-display-sourcetext-soft"] =
3458 function s2v_hookDisplay (e)
3459 {
3460 var source2View = console.views.source2;
3461
3462 source2View.unmarkHighlight();
3463
3464 var sourceTab = source2View.addSourceText (e.sourceText);
3465 if (e.rangeStart)
3466 {
3467 source2View.highlightStart = e.rangeStart;
3468 source2View.highlightEnd = e.rangeEnd;
3469 source2View.highlightTab = sourceTab;
3470 source2View.markHighlight();
3471 }
3472
3473 source2View.scrollTabTo (sourceTab, e.targetLine, 0);
3474 }
So it is hooked by an event, which contains its own sourceText. OK, so what exactly does sourceText contain? I couldn't just do a find, since there are too many occurrence of sourceText in the code. Before I give it up for the day, I decided to have a close look at the source2 code, and guess what I found in the following function!
2466 console.views.source2.getContext =
2467 function s2v_getcontext (cx)
2468 {
2469 var source2View = console.views.source2;
2470
2471 cx.lineIsExecutable = false;
2472 var sourceText;
2473
2474 if (document.popupNode.localName == "tab")
2475 {
2476 cx.index = source2View.getIndexOfTab(document.popupNode);
2477 sourceText = source2View.sourceTabList[cx.index].sourceText;
2478 cx.url = cx.urlPattern = sourceText.url;
2479 }
2480 else
2481 {
2482 cx.index = source2View.tabs.selectedIndex;
2483 sourceText = source2View.sourceTabList[cx.index].sourceText;
2484 cx.url = cx.urlPattern = sourceText.url;
2485
2486 var target = document.popupNode;
2487 while (target)
2488 {
2489 if (target.localName == "line")
2490 break;
2491 target = target.parentNode;
2492 }
2493
2494 if (target)
2495 {
2496 cx.lineNumber = parseInt(target.childNodes[1].firstChild.data);
2497
2498 var row = cx.lineNumber - 1;
2499
2500 if (sourceText.lineMap && sourceText.lineMap[row] & LINE_BREAKABLE)
2501 {
2502 cx.lineIsExecutable = true;
2503 if ("scriptInstance" in sourceText)
2504 {
2505 var scriptInstance = sourceText.scriptInstance;
2506 var scriptWrapper =
2507 scriptInstance.getScriptWrapperAtLine(cx.lineNumber);
2508 if (scriptWrapper && scriptWrapper.jsdScript.isValid)
2509 {
2510 cx.scriptWrapper = scriptWrapper;
2511 cx.pc =
2512 scriptWrapper.jsdScript.lineToPc(cx.lineNumber,
2513 PCMAP_SOURCETEXT);
2514 }
2515 }
2516 else if ("scriptWrapper" in sourceText &&
2517 sourceText.scriptWrapper.jsdScript.isValid)
2518 {
2519 cx.scriptWrapper = sourceText.scriptWrapper;
2520 cx.pc =
2521 cx.scriptWrapper.jsdScript.lineToPc(cx.lineNumber,
2522 PCMAP_PRETTYPRINT);
2523 }
2524 }
2525
2526 if (sourceText.lineMap && sourceText.lineMap[row] & LINE_BREAK)
2527 {
2528 cx.hasBreak = true;
2529 if ("scriptInstance" in sourceText)
2530 {
2531 cx.breakWrapper =
2532 sourceText.scriptInstance.getBreakpoint(cx.lineNumber);
2533 }
2534 else if ("scriptWrapper" in sourceText && "pc" in cx)
2535 {
2536 cx.breakWrapper =
2537 sourceText.scriptWrapper.getBreakpoint(cx.pc);
2538 }
2539
2540 if ("breakWrapper" in cx && cx.breakWrapper.parentBP)
2541 cx.hasFBreak = true;
2542 }
2543 else if (sourceText.lineMap && sourceText.lineMap[row] & LINE_FBREAK)
2544 {
2545 cx.hasFBreak = true;
2546 cx.breakWrapper = getFutureBreakpoint(cx.url, cx.lineNumber);
2547 }
2548 }
2549 }
2550
2551 return cx;
2552 }
getContext is probably written for getting the right context menu. User got to right click on the script for it to show up, right? Hum... for a different line, say having a break point or not, it should probably show a different menu item. Therefore, it's gotta have something to do with the real sourceText! The code confirmed my theory! Line 2500 showed it all "sourceText.lineMap[row]". So it contains an attribute called lineMap, and lineMap contains the mapping of line number and actual code for this line!!! This is all cool, but still does explain how it is populated in the xultree... ;-( Oh, wait! Something else caught my attention... The checking of sourceText is to see if it conains a scriptInstance at line 2503, and or a scriptWrapper at line 2516
"scriptInstance" in sourceText

"scriptWrapper" in sourceText
leads me to believe that somehow, one of them or both of them are important when it comes to xultree.

Just a little side track, I found out that scriptWrapper is less than scriptInstance.
2506                     var scriptWrapper =
2507 scriptInstance.getScriptWrapperAtLine(cx.lineNumber);
Now I recall that I have read something about scriptWrapper and scriptInstance in the mozilla development wiki.
http://developer.mozilla.org/en/docs/Venkman_Internals#ScriptWrapper
And this led me to http://lxr.mozilla.org/mozilla/source/extensions/venkman/resources/content/venkman-debugger.js#914. The above document explains pretty clearly how scriptWrapper works. ;-) With the pointed out source, it's even more clear. So I will just continue to move on to my quest... How did the script get to be loaded inside a xultree.

I have a feeling that I dug a little too deep, and I must have missed something, since I am already down to the scriptWrapper level, which has already passed the point where the scripts are actually being loaded, so I went back to the declaration of the source2. This time, in detail... and I easy spotted out this array.
2248     this.cmdary =
2249 [
2250 ["close-source-tab", cmdCloseTab, CMD_CONSOLE],
2251 ["find-string", cmdFindString, CMD_CONSOLE],
2252 ["save-source-tab", cmdSaveTab, CMD_CONSOLE],
2253 ["reload-source-tab", cmdReloadTab, CMD_CONSOLE],
2254 ["source-coloring", cmdToggleColoring, 0],
2255 ["toggle-source-coloring", "source-coloring toggle", 0]
2256 ];
Notice the command that's called "reload-source-tab", and there is no command called "load-source-tab" ;-) Well, I don't quite care how come there isn't a "load-source-tab" method, as long as the "reload-source-tab" tells me how it is done, I will be happy ;-)

So I looked up cmdReloadTab, and in the very end of the function, I saw
sourceTab.sourceText.reloadSource(cb);
Well, we are back at sourceText again. ;-( Maybe we over looked at this eariler. Time to check out the sourceText. I tried and tried, I suddenly got use to using the mxr site, and found out how to find a function's declaration. Looking up code is becoming a breeze for me now. ;-)

Without much effort, I found the right SourceText's declaration in venkman-static.js. It is truly the starting point of the venkman ;-) (The all mighty console object is declared there ;-))
1108 function SourceText (scriptInstance)
1109 {
Of course, there is the
1191 SourceText.prototype.reloadSource =
1192 function st_reloadsrc (cb)
1193 {
Which calls
1313 SourceText.prototype.loadSource =
1314 function st_loadsrc (cb)
1315 {
1316 var sourceText = this;
1317
1318 function onComplete (data, url, status)
1319 {
1320 //dd ("loaded " + url + " with status " + status + "n" + data);
1321 sourceText.onSourceLoaded (data, url, status);
1322 };
1323
1324 if (this.isLoaded)
1325 {
1326 /* if we're loaded, callback right now, and return. */
1327 cb (Components.results.NS_OK);
1328 return;
1329 }
1330
1331 if ("isLoading" in this)
1332 {
1333 /* if we're in the process of loading, make a note of the callback, and
1334 * return. */
1335 if (typeof cb == "function")
1336 this.loadCallbacks.push (cb);
1337 return;
1338 }
1339 else
1340 this.loadCallbacks = new Array();
1341
1342 if (typeof cb == "function")
1343 this.loadCallbacks.push (cb);
1344 this.isLoading = true;
1345
1346 var ex;
1347 var src;
1348 var url = this.url;
1349 if (url.search (/^javascript:/i) == 0)
1350 {
1351 src = url;
1352 delete this.isLoading;
1353 }
1354 else
1355 {
1356 try
1357 {
1358 loadURLAsync (url, { onComplete: onComplete });
1359 return;
1360 }
1361 catch (ex)
1362 {
1363 display (getMsg(MSN_ERR_SOURCE_LOAD_FAILED, [url, ex]), MT_ERROR);
1364 onComplete (src, url, Components.results.NS_ERROR_FAILURE);
1365 return;
1366 }
1367 }
1368
1369 onComplete (src, url, Components.results.NS_OK);
1370 }
Great! see the critical function loadURLAsync? Oh, well, before I dig too deep and get so excited about not too closely related code again, I did also see the call back of onComplete. It turns out that onComplete did the dirty work of parsing the scripts by line and pack them into sourceText.lines.
1207 SourceText.prototype.onSourceLoaded =
1208 function st_oncomplete (data, url, status)
1209 {
...
1267 var ary = data.split(/rn|n|r/m);
1268
...

1287
1288 this.lines = ary;
...
Hum... How come no trace of xultree is found? This leads me to believe that my assumption is wrong. I went back and scan through all of the code that I have just looked, and arrived back at
3040 console.views.source2.loadSourceTextAtIndex =
3041 function s2v_loadsource (sourceText, index)
3042 {
3043 var sourceTab;
3044
3045 if (index in this.sourceTabList)
3046 {
3047 sourceTab = this.sourceTabList[index];
3048 sourceTab.content = null;
3049 sourceTab.sourceText = sourceText;
3050
3051 if ("currentContent" in this)
3052 {
3053 if (!ASSERT(sourceTab.tab,
3054 "existing sourcetab not fully initialized"))
3055 {
3056 return null;
3057 }
3058 sourceTab.tab.label.setAttribute("value", sourceText.shortName);
3059 sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
3060 sourceTab.iframe.setAttribute("raiseOnSync", "true");
3061 this.syncOutputFrame(sourceTab.iframe);
3062 }
3063
3064 }
3065 else
3066 {
3067 sourceTab = {
3068 sourceText: sourceText,
3069 tab: null,
3070 iframe: null,
3071 content: null
3072 };
3073
3074 this.sourceTabList[index] = sourceTab;
3075
3076 if ("currentContent" in this)
3077 {
3078 this.createFrameFor(sourceTab, index, true);
3079 }
3080 }
3081
3082 return sourceTab;
3083 }
So this function is really trying to load source text into a tag with index defined by the index variable. If you take a closer look, you will find
3058             sourceTab.tab.label.setAttribute("value", sourceText.shortName);
3059 sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
3060 sourceTab.iframe.setAttribute("raiseOnSync", "true");
So it is an iframe, and the file is loaded through sourceText.jsdURL. So it is not an xultree like other views, but an iframe!!! And of cource, there are utilities to create frame
console.views.source2.createFrameFor
and update margins
console.views.source2.updateLineMargin
etc, etc. Phew... My initial xultree assumption was wrong, and it is implemented using iframe.

Just to make the quest more complete, I am including how firebug is implementing the source text area ;-) This didn't take me nearly as long to figure out.

in debugger.js' ScriptPanel class, you will find a function called createSourceBox.
createSourceBox: function(sourceFile)
{
var lines = loadScriptLines(sourceFile, this.context);
if (!lines)
return null;

var maxLineNoChars = (lines.length + "").length;

var sourceBox = this.document.createElement("div");
sourceBox.repObject = sourceFile;
setClass(sourceBox, "sourceBox");
collapse(sourceBox, true);
this.panelNode.appendChild(sourceBox);

// For performance reason, append script lines in large chunks using the throttler,
// otherwise displaying a large script will freeze up the UI
var min = 0;
do
{
var max = min + scriptBlockSize;
if (max > lines.length)
max = lines.length;

var args = [lines, min, max-1, maxLineNoChars, sourceBox];
this.context.throttle(appendScriptLines, top, args);

min += scriptBlockSize;
} while (max < lines.length);

this.context.throttle(setLineBreakpoints, top, [sourceFile, sourceBox]);

if (sourceFile.text)
this.anonSourceBoxes.push(sourceBox);
else
this.sourceBoxes[sourceFile.href] = sourceBox;

return sourceBox;
},
OK! That was a long day for me... ;-)

1 comment:

Caio Tiago said...

Great post.

Sometimes I like a lot the way Venkman and DOM Inspector works.

I know their history and such.

Well... Venkman was completely broken for some time on trunk and some developers went to help it back to work (another tier one issue appeared and last a while, but was resolved before 1.9).

But I got the feel that the Venkman and the DOMi codes are more mature than Firebug code.
Sometimes I think that it would be easier to use both and add functionality.