Javascript Source View II - Firebug vs Venkman

OK, last time, I was very happy to find out that venkman was using the XUL tree to represent the javascript source, while firebug was using html <a> tag and <span> tags.

As I was trying to find the script to load into the source view, I started to wonder how did those two software did it. I would imagine that we need to implement something that will

1. load the file into a stream
2. process the stream line by line

The findings from venkman actually amazed me once again, and along the way, I picked up a few other nice things ;-) So here it goes...

First, I was quite confident that I want to do it in xul tree way instead of the html way, so I started to scratch my head and buried my head into the venkman source code once again ;-) But before I went too far, I discovered that there are actually two different source views in venkman! One is called source, another one is called source2. source2 is the one that enables us to set breakpoints, and source doesn't do much, and it is hidden by default.

From venkman-views.xul
     <!-- source2 view -->
  
      <floatingview id="source2" title="&Source.label;" flex="1">
        <vbox flex="1">
          <tabbox id="source2-tabbox" flex="1">
            <tabs id="source2-tabs">
              <!--
                We've got to put this placeholder tab in the tabs element to avoid
                bogus strict warnings and exceptions.
              -->
              <tab id="source2-bloke" hidden="true"/>
            </tabs>
            <tabpanels id="source2-deck" flex="1"/>
          </tabbox>
          <textbox class="plain" id="source2-heading" readonly="true"/>
        </vbox>
      </floatingview>
  
      <!-- source view -->
  
      <floatingview id="source" title="&Source.label;" flex="1">
        <vbox id="source-view-content" flex="1">
          <toolbox>
            <toolbar id="source-header" grippytooltiptext="&SourceHeader.tip;">
              <label id="source-url" flex="1" crop="end"/>
            </toolbar>
          </toolbox>
          <tree id="source-tree" flex="1" persist="width"
            class="focusring"
            onclick="console.views.source.onClick(event);"
            onselect="console.views.source.onSelect(event);"
            context="context:source">
  
            <treecols>
              <treecol id="source:col-0" width="20px"
                display="&SourceCol0.display;" persist="hidden width"/>
              <splitter class="tree-splitter"/>
              <treecol id="source:col-1" width="50px"
                display="&SourceCol1.display;" persist="hidden width"/>
              <splitter class="tree-splitter"/>
              <treecol id="source:col-2" flex="1" display=""
                ignoreincolumnpicker="true" persist="hidden width"/>
            </treecols>
  
            <treechildren id="source-tree-body"/>
  
          </tree>
        </vbox>
      </floatingview>

From venkman-views.js

 /*******************************************************************************
  * Source2 View
  *******************************************************************************/
 
 console.views.source2 = new Object();
 
 const VIEW_SOURCE2 = "source2";
 console.views.source2.viewId = VIEW_SOURCE2;
 
 console.views.source2.init =
 function ss_init ()
 {

 /*******************************************************************************
  * Source View
  *******************************************************************************/
 console.views.source = new BasicOView();
 
 const VIEW_SOURCE = "source";
 console.views.source.viewId = VIEW_SOURCE;
 
 console.views.source.details = null;
 console.views.source.prettyPrint = false;
 
 console.views.source.init =
 function sv_init()
 {

Notice that Source2 is an Object, while Source is a BasicOView. As what I usually do, I first scan the code for possible candidates of the key methods that I am looking for, and these two methods caught my attention.

 console.views.source2.loadSourceTextAtIndex =
 function s2v_loadsource (sourceText, index)
 {
     var sourceTab;
    
     if (index in this.sourceTabList)
     {
         sourceTab = this.sourceTabList[index];
         sourceTab.content = null;
         sourceTab.sourceText = sourceText;
        
         if ("currentContent" in this)
         {
             if (!ASSERT(sourceTab.tab,
                         "existing sourcetab not fully initialized"))
             {
                 return null;
             }
             sourceTab.tab.label.setAttribute("value", sourceText.shortName);
             sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
             sourceTab.iframe.setAttribute("raiseOnSync", "true");
             this.syncOutputFrame(sourceTab.iframe);
         }
        
     }
     else
     {
         sourceTab = {
             sourceText: sourceText,
             tab: null,
             iframe: null,
             content: null
         };
 
         this.sourceTabList[index] = sourceTab;
    
         if ("currentContent" in this)
         {
             this.createFrameFor(sourceTab, index, true);
         }
     }
 
     return sourceTab;
 }

 console.views.source2.addSourceText =
 function s2v_addsource (sourceText)
 {

addSourceText calls loadSourceTextAtIndex to load a specific javascript source into source2's tabbed source views. addSourceText function will just happily take the sourceText, and stash it into a tab ;-) Great, the sourceText explains it all! (Grrr...) So what the heck is sourceText? For a while, I thought it was real string, but I quickly had an impression that I am wrong, because strings are not smart enough to be displayed into a xultree. So who calls the addSourceText? I dig on...

 console.views.source2.hooks["hook-display-sourcetext"] =
 console.views.source2.hooks["hook-display-sourcetext-soft"] =
 function s2v_hookDisplay (e)
 {
     var source2View = console.views.source2;
 
     source2View.unmarkHighlight();
    
     var sourceTab = source2View.addSourceText (e.sourceText);
     if (e.rangeStart)
     {
         source2View.highlightStart = e.rangeStart;
         source2View.highlightEnd = e.rangeEnd;
         source2View.highlightTab = sourceTab;
         source2View.markHighlight();
     }
    
     source2View.scrollTabTo (sourceTab, e.targetLine, 0);
 }

So it is hooked by an event, which contains its own sourceText. OK, so what exactly does sourceText contain? I couldn't just do a find, since there are too many occurrence of sourceText in the code. Before I give it up for the day, I decided to have a close look at the source2 code, and guess what I found in the following function!

 console.views.source2.getContext =
 function s2v_getcontext (cx)
 {
     var source2View = console.views.source2;
    
     cx.lineIsExecutable = false;
     var sourceText;
 
     if (document.popupNode.localName == "tab")
     {
         cx.index = source2View.getIndexOfTab(document.popupNode);
         sourceText = source2View.sourceTabList[cx.index].sourceText;
         cx.url = cx.urlPattern = sourceText.url;
     }
     else
     {
         cx.index = source2View.tabs.selectedIndex;
         sourceText = source2View.sourceTabList[cx.index].sourceText;
         cx.url = cx.urlPattern = sourceText.url;
 
         var target = document.popupNode;
         while (target)
         {
             if (target.localName == "line")
                 break;
             target = target.parentNode;
         }
 
         if (target)
         {
             cx.lineNumber = parseInt(target.childNodes[1].firstChild.data);
 
             var row = cx.lineNumber - 1;
 
             if (sourceText.lineMap && sourceText.lineMap[row] & LINE_BREAKABLE)
             {
                 cx.lineIsExecutable = true;
                 if ("scriptInstance" in sourceText)
                 {
                     var scriptInstance = sourceText.scriptInstance;
                     var scriptWrapper =
                         scriptInstance.getScriptWrapperAtLine(cx.lineNumber);
                     if (scriptWrapper && scriptWrapper.jsdScript.isValid)
                     {
                         cx.scriptWrapper = scriptWrapper;
                         cx.pc =
                             scriptWrapper.jsdScript.lineToPc(cx.lineNumber,
                                                              PCMAP_SOURCETEXT);
                     }
                 }
                 else if ("scriptWrapper" in sourceText &&
                          sourceText.scriptWrapper.jsdScript.isValid)
                 {
                     cx.scriptWrapper = sourceText.scriptWrapper;
                     cx.pc =
                         cx.scriptWrapper.jsdScript.lineToPc(cx.lineNumber,
                                                             PCMAP_PRETTYPRINT);
                 }
             }
            
             if (sourceText.lineMap && sourceText.lineMap[row] & LINE_BREAK)
             {
                 cx.hasBreak = true;
                 if ("scriptInstance" in sourceText)
                 {
                     cx.breakWrapper =
                         sourceText.scriptInstance.getBreakpoint(cx.lineNumber);
                 }
                 else if ("scriptWrapper" in sourceText && "pc" in cx)
                 {
                     cx.breakWrapper =
                         sourceText.scriptWrapper.getBreakpoint(cx.pc);
                 }
                
                 if ("breakWrapper" in cx && cx.breakWrapper.parentBP)
                     cx.hasFBreak = true;
             }
             else if (sourceText.lineMap && sourceText.lineMap[row] & LINE_FBREAK)
             {
                 cx.hasFBreak = true;
                 cx.breakWrapper = getFutureBreakpoint(cx.url, cx.lineNumber);
             }
         }
     }   
    
     return cx;
 }

getContext is probably written for getting the right context menu. User got to right click on the script for it to show up, right? Hum... for a different line, say having a break point or not, it should probably show a different menu item. Therefore, it's gotta have something to do with the real sourceText! The code confirmed my theory! Line 2500 showed it all "sourceText.lineMap[row]". So it contains an attribute called lineMap, and lineMap contains the mapping of line number and actual code for this line!!! This is all cool, but still does explain how it is populated in the xultree... ;-( Oh, wait! Something else caught my attention... The checking of sourceText is to see if it conains a scriptInstance at line 2503, and or a scriptWrapper at line 2516

"scriptInstance" in sourceText

"scriptWrapper" in sourceText

leads me to believe that somehow, one of them or both of them are important when it comes to xultree.

Just a little side track, I found out that scriptWrapper is less than scriptInstance.

                     var scriptWrapper =
                         scriptInstance.getScriptWrapperAtLine(cx.lineNumber);

Now I recall that I have read something about scriptWrapper and scriptInstance in the mozilla development wiki.

http://developer.mozilla.org/en/docs/Venkman_Internals#ScriptWrapper

And this led me to http://lxr.mozilla.org/mozilla/source/extensions/venkman/resources/content/venkman-debugger.js#914. The above document explains pretty clearly how scriptWrapper works. ;-) With the pointed out source, it's even more clear. So I will just continue to move on to my quest... How did the script get to be loaded inside a xultree.

I have a feeling that I dug a little too deep, and I must have missed something, since I am already down to the scriptWrapper level, which has already passed the point where the scripts are actually being loaded, so I went back to the declaration of the source2. This time, in detail... and I easy spotted out this array.

     this.cmdary =
         [
          ["close-source-tab",          cmdCloseTab,              CMD_CONSOLE],
          ["find-string",               cmdFindString,            CMD_CONSOLE],
          ["save-source-tab",           cmdSaveTab,               CMD_CONSOLE],
          ["reload-source-tab",         cmdReloadTab,             CMD_CONSOLE],
          ["source-coloring",           cmdToggleColoring,                  0],
          ["toggle-source-coloring",    "source-coloring toggle",           0]
         ];

Notice the command that's called "reload-source-tab", and there is no command called "load-source-tab" ;-) Well, I don't quite care how come there isn't a "load-source-tab" method, as long as the "reload-source-tab" tells me how it is done, I will be happy ;-)

So I looked up cmdReloadTab, and in the very end of the function, I saw

sourceTab.sourceText.reloadSource(cb);

Well, we are back at sourceText again. ;-( Maybe we over looked at this eariler. Time to check out the sourceText. I tried and tried, I suddenly got use to using the mxr site, and found out how to find a function's declaration. Looking up code is becoming a breeze for me now. ;-)

Without much effort, I found the right SourceText's declaration in venkman-static.js. It is truly the starting point of the venkman ;-) (The all mighty console object is declared there ;-))

 function SourceText (scriptInstance)
 {

Of course, there is the

 SourceText.prototype.reloadSource =
 function st_reloadsrc (cb)
 {

Which calls

 SourceText.prototype.loadSource =
 function st_loadsrc (cb)
 {
     var sourceText = this;
    
     function onComplete (data, url, status)
     {
         //dd ("loaded " + url + " with status " + status + "n" + data);
         sourceText.onSourceLoaded (data, url, status);
     };
 
     if (this.isLoaded)
     {
         /* if we're loaded, callback right now, and return. */
         cb (Components.results.NS_OK);
         return;
     }
 
     if ("isLoading" in this)
     {
         /* if we're in the process of loading, make a note of the callback, and
          * return. */
         if (typeof cb == "function")
             this.loadCallbacks.push (cb);
         return;
     }
     else
         this.loadCallbacks = new Array();
 
     if (typeof cb == "function")
         this.loadCallbacks.push (cb);
     this.isLoading = true;   
    
     var ex;
     var src;
     var url = this.url;
     if (url.search (/^javascript:/i) == 0)
     {
         src = url;
         delete this.isLoading;
     }
     else
     {
         try
         {
             loadURLAsync (url, { onComplete: onComplete });
             return;
         }
         catch (ex)
         {
             display (getMsg(MSN_ERR_SOURCE_LOAD_FAILED, [url, ex]), MT_ERROR);
             onComplete (src, url, Components.results.NS_ERROR_FAILURE);
             return;
         }
     }
 
     onComplete (src, url, Components.results.NS_OK);
 }

Great! see the critical function loadURLAsync? Oh, well, before I dig too deep and get so excited about not too closely related code again, I did also see the call back of onComplete. It turns out that onComplete did the dirty work of parsing the scripts by line and pack them into sourceText.lines.

 SourceText.prototype.onSourceLoaded =
 function st_oncomplete (data, url, status)
 {
...
     var ary = data.split(/rn|n|r/m);       
 
...
    
     this.lines = ary;
...

Hum... How come no trace of xultree is found? This leads me to believe that my assumption is wrong. I went back and scan through all of the code that I have just looked, and arrived back at

 console.views.source2.loadSourceTextAtIndex =
 function s2v_loadsource (sourceText, index)
 {
     var sourceTab;
    
     if (index in this.sourceTabList)
     {
         sourceTab = this.sourceTabList[index];
         sourceTab.content = null;
         sourceTab.sourceText = sourceText;
        
         if ("currentContent" in this)
         {
             if (!ASSERT(sourceTab.tab,
                         "existing sourcetab not fully initialized"))
             {
                 return null;
             }
             sourceTab.tab.label.setAttribute("value", sourceText.shortName);
             sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
             sourceTab.iframe.setAttribute("raiseOnSync", "true");
             this.syncOutputFrame(sourceTab.iframe);
         }
        
     }
     else
     {
         sourceTab = {
             sourceText: sourceText,
             tab: null,
             iframe: null,
             content: null
         };
 
         this.sourceTabList[index] = sourceTab;
    
         if ("currentContent" in this)
         {
             this.createFrameFor(sourceTab, index, true);
         }
     }
 
     return sourceTab;
 }

So this function is really trying to load source text into a tag with index defined by the index variable. If you take a closer look, you will find

             sourceTab.tab.label.setAttribute("value", sourceText.shortName);
             sourceTab.iframe.setAttribute("targetSrc", sourceText.jsdURL);
             sourceTab.iframe.setAttribute("raiseOnSync", "true");

So it is an iframe, and the file is loaded through sourceText.jsdURL. So it is not an xultree like other views, but an iframe!!! And of cource, there are utilities to create frame

console.views.source2.createFrameFor

and update margins

console.views.source2.updateLineMargin

etc, etc. Phew... My initial xultree assumption was wrong, and it is implemented using iframe.

Just to make the quest more complete, I am including how firebug is implementing the source text area ;-) This didn't take me nearly as long to figure out.

in debugger.js' ScriptPanel class, you will find a function called createSourceBox.

createSourceBox: function(sourceFile)
{
var lines = loadScriptLines(sourceFile, this.context);
if (!lines)
return null;

var maxLineNoChars = (lines.length + "").length;

var sourceBox = this.document.createElement("div");
sourceBox.repObject = sourceFile;
setClass(sourceBox, "sourceBox");
collapse(sourceBox, true);
this.panelNode.appendChild(sourceBox);

// For performance reason, append script lines in large chunks using the throttler,
// otherwise displaying a large script will freeze up the UI
var min = 0;
do
{
var max = min + scriptBlockSize;
if (max > lines.length)
max = lines.length;

var args = [lines, min, max-1, maxLineNoChars, sourceBox];
this.context.throttle(appendScriptLines, top, args);

min += scriptBlockSize;
} while (max < lines.length);

this.context.throttle(setLineBreakpoints, top, [sourceFile, sourceBox]);

if (sourceFile.text)
this.anonSourceBoxes.push(sourceBox);
else
this.sourceBoxes[sourceFile.href] = sourceBox;

return sourceBox;
},

OK! That was a long day for me... ;-)