canmove, Confirmed users
1,953
edits
MDN/Archives/Kuma/Scripting: Difference between revisions
m
Jswisher moved page MDN/Kuma/Scripting to MDN/Archives/Kuma/Scripting
LesOrchard (talk | contribs) No edit summary |
m (Jswisher moved page MDN/Kuma/Scripting to MDN/Archives/Kuma/Scripting) |
||
| (28 intermediate revisions by one other user not shown) | |||
| Line 19: | Line 19: | ||
* Local API with access to Wiki data queries (eg. custom tables of contents, etc) | * Local API with access to Wiki data queries (eg. custom tables of contents, etc) | ||
* Network API with access to external web services (eg. RSS feeds, bugzilla, etc) | * Network API with access to external web services (eg. RSS feeds, bugzilla, etc) | ||
* L10N-aware constructs and alternate content | |||
** Example: [https://developer.mozilla.org/Template:XULAttrInheritedWide Template:XULAttrInheritedWide] | |||
** eg. span lang="en" class="lang lang-en", span lang="de" class="lang lang-de", span lang="*" class="lang lang-*" | |||
* Safe for use by wiki content authors | * Safe for use by wiki content authors | ||
** (Not entirely sure what makes it safe, need more research) | ** (Not entirely sure what makes it safe, need more research) | ||
| Line 69: | Line 72: | ||
== Miscellaneous notes == | == Miscellaneous notes == | ||
* [http://nodejs.org/docs/v0.3.1/api/vm.html#vm.runInNewContext Built-in node.js sandboxing features] | * Would like a relatively boring solution that matches core competencies | ||
* [http://gf3.github.com/sandbox/ Sandbox] to wrap JS execution in node.js? | ** Components should have active existing communities | ||
** Seems to have some more features than built-in node.js sandboxing | ** Try to stay close to dev team core-competencies | ||
** timeouts, restricted method access, graceful errors | ** Try to make it sensible to host for care & feeding by IT | ||
* Switch from Lua to JavaScript basis | |||
** JavaScript is more of a webdev core-competency | |||
** JS is pretty close to Lua | |||
** node.js has an active community, works out-of-the-box | |||
*** Maybe consider [https://github.com/zpao/spidernode SpiderNode], but only if we get benefits beyond not-invented-here purity | |||
* Or, do something ''with'' Lua? | |||
** [http://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2012-01-30/Technology_report Wikipedia seems fond of Lua for ''their'' templates] | |||
* HTTP proxy service vs embedding JS | |||
** Embedding JS in a Python app seems experimental, and outside webdev team core-competency | |||
** HTTP is a webdev core-compentency, and has loose-coupling and RESTafarian benefits | |||
** Taming a code execution environment in its own process seems safer and more easily scaled | |||
* Taming semi- or untrusted JavaScript code | |||
** [http://nodejs.org/docs/v0.3.1/api/vm.html#vm.runInNewContext Built-in node.js sandboxing features] | |||
** [http://gf3.github.com/sandbox/ Sandbox] to wrap JS execution in node.js? | |||
*** Seems to have some more features than built-in node.js sandboxing | |||
*** timeouts, restricted method access, graceful errors | |||
* Use standalone [https://github.com/visionmedia/ejs embedded JS templates] instead of literals embedded in script? | * Use standalone [https://github.com/visionmedia/ejs embedded JS templates] instead of literals embedded in script? | ||
** Employs code-in-markup instead of markup-in-code, and avoids the need for fancy compiler hijinx | ** Employs code-in-markup instead of markup-in-code, and avoids the need for fancy compiler hijinx | ||
** This is being used in other projects | |||
* Might [https://github.com/laverdet/js-xml-literal js-xml-literals] come in handy? | * Might [https://github.com/laverdet/js-xml-literal js-xml-literals] come in handy? | ||
** [https://github.com/laverdet/js-xml-literal/issues/2 I failed on my first attempt to install], so haven't seen this in action. | ** [https://github.com/laverdet/js-xml-literal/issues/2 I failed on my first attempt to install], so haven't seen this in action. | ||
** This seems experimental; no one else really seems to be using it yet | |||
* Should we get a compiler / language expert in on this for opinions? | * Should we get a compiler / language expert in on this for opinions? | ||
** This smells like dangerous excitement | |||
* Could we port DekiScript directly? (It's open source, right?) | * Could we port DekiScript directly? (It's open source, right?) | ||
| Line 84: | Line 114: | ||
The basic concept: | The basic concept: | ||
* Kuma (Django/Python) serves as the wiki/CMS for both general content and template scripts | |||
* Document views are proxied through a node.js-based service that evaluates and executes inline macros. | |||
* Support inline parameterized '''macros''' that invoke template scripts. | * Support inline parameterized '''macros''' that invoke template scripts. | ||
** Macros are usable by all content editors | ** Macros are usable by all content editors | ||
** Macros are neutered function calls, basically, and not full JS expressions | |||
** Macros are handled by a custom parser, and definitely not just <code>eval()</code>'ed | |||
* Support '''template scripts''' that are evaluated in a node.js sandbox as [https://github.com/visionmedia/ejs EJS templates]. | * Support '''template scripts''' that are evaluated in a node.js sandbox as [https://github.com/visionmedia/ejs EJS templates]. | ||
** '''Template scripts''' could be made editable only by elevated-privilege editors | ** '''Template scripts''' could be made editable only by elevated-privilege editors | ||
Considering this from a high-level view of off-the-shelf parts that could be glued together: | Considering this from a high-level view of off-the-shelf parts that could be glued together: | ||
| Line 97: | Line 129: | ||
* [https://github.com/visionmedia/ejs EJS templates] | * [https://github.com/visionmedia/ejs EJS templates] | ||
* Convenience methods to perform wiki content queries and utility functions | * Convenience methods to perform wiki content queries and utility functions | ||
** Something like what [http://developer.mindtouch.com/en/docs/DekiScript/Reference/Wiki_Functions_and_Variables DekiScript offers for wiki functions and variables] | |||
* Plentiful and intelligent HTTP-based caching | * Plentiful and intelligent HTTP-based caching | ||
' | === Prototype === | ||
lorchard has started work on a prototype in node.js: | |||
* https://github.com/lmorchard/kumascript | |||
=== Flow sketch === | |||
Here is a rough sketch of the process and network flows involved in this proposed solution: | |||
[[File:Kumascript.png|1000px|Kuma script flow]] | |||
More notes: | |||
* If the caching works right, we should be able to almost never load & compile templates unless they're edited. (Cache invalidation? HEAD requests before GET requests?) | |||
* When a Kuma wiki page is edited, the last-cached KumaScript response is discarded. | |||
* To support preview functionality, we can accept HTTP POST of arbitrary page source at the start of the KumaScript flow. Not cached, which is a feature. | |||
=== Code samples (imaginary) === | === Code samples (imaginary) === | ||
| Line 154: | Line 201: | ||
* Can this be done in a way that restricts file, network, memory, and CPU usage? | * Can this be done in a way that restricts file, network, memory, and CPU usage? | ||
** Anything else dangerous and in need of restriction? | ** Anything else dangerous and in need of restriction? | ||
** How restrictive do we need to be? (ie. is trusting a certain set of authors enough?) | |||
** How restrictive is DekiScript? | |||
* Options inside node.js | * Options inside node.js | ||
** node.js has sandboxing out-of-the-box | ** [http://nodejs.org/docs/v0.3.1/api/vm.html#vm.runInNewContext node.js has sandboxing out-of-the-box] | ||
** There's also [http://gf3.github.com/sandbox/ Sandbox] | ** There's also [http://gf3.github.com/sandbox/ Sandbox] | ||
* Options for host running node.js | * Options for host running node.js | ||
** No filesystem access at all (chroot?) | ** No filesystem access at all (chroot?) | ||
*** and/or [http://lxc.sourceforge.net/ LXC] | *** and/or [http://lxc.sourceforge.net/ LXC] | ||
** Whitelisted network access (firewall rules?) | ** Whitelisted network access (eg. firewall rules? limit base URLs of services?) | ||
** Limited execution time (kill the process after | ** Limited execution time (eg. kill the process after 30000 msec?) | ||
** Limited memory usage (kill the process after 10MB consumed?) | ** Limited memory usage (eg. kill the process after 10MB consumed?) | ||
** Auto-disable script if abuse detected? | ** Auto-disable script if abuse detected (eg. penalty box for X minutes?) | ||