Gaia/Build/RefactoringToNodejs

Goal

Build system has run on XULRunner(XPCShell) since a long time ago, also requiring Make and shell script to shuffle, optimize or compress files around. However, all features are able to be done by Node.js so that everyone can benefit from the good part of Node.js. Our plan is rewrite/refactoring existed XULRunner-style JavaScript into Node.js.

Profit from Node.js

• Web developers are familiar with JavaScript, also Node.js is the most popular and acceptable for those who are familiar with client side JavaScript. It could attract more contributors to involve and improve Firefox OS.
• NPM is a well-known build-in package manager of node, such tool is commonly used for web developers for years. It will make your life easier that developers can install third-party libraries and prevent us to reinvent the wheel if there is no such library provided by XULRunner.
• It would be great if we build up a pure JS system. To re-implement Makefile and shell script is a hard work, NPM provides a bunch of packages that let us build our pure JS system more quickly, flexible and maintainable.

Plan

Node.js refactoring plan will split into 4 stages.

Stage 0

Goal: Enable test suites for node build script

For finding out potential risks during node refactoring, we're going to enable build test suites for node.js beforehand, namely (Gbn/Gbun) in Treeherder and target on gaia-try, try and b2g-inbound. After that, the light of Gbn and Gbun will keep red for a while until stage 1 is finished.

Milestone

• Add RUN_ON_NODE switcher in Makefile for enabling node.js build script.
• Enable Gbn in Treeherder (gaia-try / try / b2g-inbound)
• Enable Gbun in Treeherder (gaia-try / try / b2g-inbound)

Stage 1

Goal: Complete the implementation of node build scripts that coexist with XPCShell

Focus on refactoring build system core functionality (parallel build, webapp-manifests, preferences, webapp-optimize, webapp-zip... etc). All of them exist certain specific code only use for XPCShell. Unfinished Node.js build script (build/node-utils.js) is already located in Gaia for a while (See also: bug 955988. We should keep working on it and keep both Node and XPCShell compatible in build system.

Stage 2 (Draft)

Goal: Using NPM modules to build up a pure JS build system instead of Makefile.

In stage 2, we're going to rewrite Makefile into JS module. Currently, we'll choose Gulp.js for our new build system. In order to rewrite Makefile into JS module, Gulp's incremental builds and stream workflow seems to meet our needs. I'm looking forward to seeing a readable and maintainable gulpfile.js in Firefox OS.

Stage 3 (Draft)

Goal: Turn on node build script by default and remove all legacy APIs.

Until stage 1 and stage 2 are stable, we're going to enable node build script by default and remove all legacy APIs.

Milestone

• Turn on node build script by default
• Refactor utils-node.js API and remove utils-xpc.js and utils.js
• Refactor all build/*.js to conform to node-style

Q & A

• WebIDE is trying to make developing Gaia more easily by clicking a button without any command line tricks. It triggers building process by invoking "make" currently. In XULRunner build system, after rewriting Makefile to pure JS module to take over it's job, WebIDE can build Gaia without any dependencies. (such as make and node)

For node build system, requiring Node.js may be a little bit inconvenient. However, it won't frustrate many users to get started with hacking Gaia since Node provides a great installer for all platforms. After Node.js installed, we can build up new features quickly by NPM and also speed up rewriting Makefile to pure JS. On the other hand, try to rewrite Makefile from scratch in XULRunner seems not so easy. I'm pretty sure and still believe that NPM ecosystem will bring lots of benefits more than XULRunner.

• Some countries have network problems when pulling packages from NPM.