MDN/Archives/Kuma/Scripting

Overview

The mission of the Kuma wiki is to replace MindTouch. One of the advanced features of MindTouch is DekiScript, a Lua-based scripting environment for wiki authors.

We'd like to replace this feature with something secure, capable, modern, and convenient. We'd also like it if we could carry over the existing body of DekiScript with minimal manual changes. Let's see what's feasible.

DekiScript highlights

• Lua-based dynamic scripting language
• XML/HTML literals for easier construction of well-formed markup
• Inline { { script expressions } } in wiki page content
• Long-form Template: scripts callable with parameters from inline expressions
• Local API with access to Wiki data queries (eg. custom tables of contents, etc)
• Network API with access to external web services (eg. RSS feeds, bugzilla, etc)
• Safe for use by wiki content authors
  • (Not entirely sure what makes it safe, need more research)
    • Seems like long-form Template: scripts are editable only by elevated-permission users?
    • Normal content editors can make calls to Template: scripts, but not employ all scripting features?

Code samples

Here's some inline script excerpted from en/Firefox_9_for_developers:

    <li>The <code>value</code> attribute of {{HTMLElement("li")}} now can be
    negative as specified in HTML5. Previously negative values were converted
    to 0.</li>

Here's the source of Template:HTMLElement, a commonly used script:

    /* accepts as input one required parameter: HTML element to create a xref to */
    var uri = uri.parts(Page.uri);
    var lang = string.tolower(uri.path[0]);
    if (string.contains(lang, "project") || string.contains(lang, "Project")) {
      let lang = string.substr(lang, 8);
    }
    /* fall back to page.language on a user page */
    else if (string.StartsWith(lang, "user:")) {
      let lang = page.language;
    }
    var name = $0;
    var sectionname = "Element";

    if (!string.compare("es", string.tolower(lang))) {
      let sectionname = "Elemento";
    }

    if (args.title) {
      let name = args.title;
    }
    var dest = lang .. '/' .. 'HTML/' .. sectionname .. '/' .. name;
    var destEng = 'en/HTML/Element/' .. name;
    if (wiki.pageExists(dest)) { /* the page exists */
      <code>(web.link(wiki.uri(dest), '<' .. name .. '>'))</code>;
    } else if (lang == 'zh_tw' && wiki.pageExists(destEng)){
      /* the MozTW community consider links to English pages better than red ones. 
         I'll write about this to mozilla.dev.mdc later */   
      <code>(web.link(wiki.uri(destEng), '<' .. name .. '>'))</code>;
    } else { /* the page doesn't exist */
      var targeturi = "https://developer.mozilla.org/Article_not_found?uri=" .. dest;
      <code><a rel=('internal') href=(targeturi) class=('new')>web.text('<' .. name .. '>')</a></code>
    }

Miscellaneous notes

• Built-in node.js sandboxing features
• Sandbox to wrap JS execution in node.js?
  • Seems to have some more features than built-in node.js sandboxing
  • timeouts, restricted method access, graceful errors
• Use standalone embedded JS templates instead of literals embedded in script?
  • Employs code-in-markup instead of markup-in-code, and avoids the need for fancy compiler hijinx
• Might js-xml-literals come in handy?
  • I failed on my first attempt to install, so haven't seen this in action.
• Should we get a compiler / language expert in on this for opinions?
• Could we port DekiScript directly? (It's open source, right?)

Proposed solution #1

The basic concept:

• Support inline parameterized macros that invoke template scripts.
  • Macros are usable by all content editors
• Support template scripts that are evaluated in a node.js sandbox as EJS templates.
  • Template scripts could be made editable only by elevated-privilege editors
• Kuma (Django/Python) serves as the wiki/CMS
• Document views are proxied through a node.js-based service that evaluates and executes inline macros.

Considering this from a high-level view of off-the-shelf parts that could be glued together:

• Kuma wiki CMS (Django/Python)
• Sandboxed server-side JavaScript filter proxy (node.js)
• EJS templates
• Convenience methods to perform wiki content queries and utility functions
• Plentiful and intelligent HTTP-based caching

DRAWRINGS GO HERE

Code samples (imaginary)

We can limit inline expressions to invoking long-form templates, rather than free-form scripting. Usage still looks familiar with respect to content "in the wild", though:

    <li>The <code>value</code> attribute of {{HTMLElement("li")}} now can be
    negative as specified in HTML5. Previously negative values were converted
    to 0.</li>

Long-form templates become embedded JS templates, something like this:

<%
    /* accepts as input one required parameter: HTML element to create a xref to */
    var uri = uri.parts(Page.uri);
    var lang = string.tolower(uri.path[0]);
    if (string.contains(lang, "project") || string.contains(lang, "Project")) {
      let lang = string.substr(lang, 8);
    }
    /* fall back to page.language on a user page */
    else if (string.StartsWith(lang, "user:")) {
      let lang = page.language;
    }
    var name = arguments[0];
    var sectionname = "Element";

    if (!string.compare("es", string.tolower(lang))) {
      sectionname = "Elemento";
    }

    if (args.title) {
      name = args.title;
    }
    var dest = lang + '/' + 'HTML/' + sectionname + '/' + name;
    var destEng = 'en/HTML/Element/' + name;
    if (wiki.pageExists(dest)) { /* the page exists */
      %> <code><%- web.link(wiki.uri(dest), '<' + name + '>')) %></code> <%
    } else if (lang == 'zh_tw' && wiki.pageExists(destEng)){
      /* the MozTW community consider links to English pages better than red ones. 
         I'll write about this to mozilla.dev.mdc later */   
      %> <code><%- web.link(wiki.uri(destEng), '<' + name + '>')) %></code> <%;
    } else { /* the page doesn't exist */
      var targeturi = "https://developer.mozilla.org/Article_not_found?uri=" .. dest;
      %> <code><a rel="internal" href="<%= targeturi %>" class="new"><%- web.text('<' + name + '>') %></a></code> <%
    }
%>

The above presupposes an API similar to what's exposed to DekiScript, is non-functional, and mainly serves as a thought experiment in comparative syntax.

Sandboxed JavaScript execution

• Can this be done in a way that restricts file, network, memory, and CPU usage?
  • Anything else dangerous and in need of restriction?
• Options inside node.js
  • node.js has sandboxing out-of-the-box, and there's Sandbox
  • There's also Sandbox
• Options for host running node.js
  • No filesystem access at all (chroot?)
    • and/or LXC
  • Whitelisted network access (firewall rules?)
  • Limited execution time (kill the process after 30 sec?)
  • Limited memory usage (kill the process after 10MB consumed?)
  • Auto-disable script if abuse detected?