Gaia/Build/RefactoringToNodejs: Difference between revisions

Jump to navigation Jump to search

Gaia/Build/RefactoringToNodejs (view source)

Revision as of 07:22, 3 February 2015

488 bytes removed ,  3 February 2015
Edits
(Grammar, terms etc.)
(Edits)
Line 1: Line 1:
= Goal =
Build system has run on XULRunner(XPCShell) since a long time ago, also requiring Make and shell script to shuffle, optimize or compress files around. However, all features are able to be done by [http://nodejs.org/ Node.js] so that everyone can benefit from the good part of [http://nodejs.org/ Node.js]. Our plan is rewrite/refactoring existed XULRunner-style JavaScript into [http://nodejs.org/ Node.js].
= Profit from [http://nodejs.org/ Node.js] =
* Easier to catch up: Web developers are familiar with JavaScript, also [http://nodejs.org/ Node.js] is the most popular and acceptable for those who are familiar with client side JavaScript. It could attract more contributors to involve and improve Firefox OS.
* Abundant tool: NPM is a well-known build-in package manager of node, such tool is commonly used for web developers for years. It will make your life easier that developers can install third-party libraries and prevent us to reinvent the wheel if there is no such library provided by XULRunner.
* Pure JS: It would be great if we build up a pure JS system. To re-implement Makefile into JS is a hard work without third-party library, NPM provides a bunch of packages that let us build our pure JS system more quickly, flexible and maintainable.
= Overviews =
= Overviews =


Line 14: Line 4:


* A lot of OS-specific detail leaks through, making modification hard to debug
* A lot of OS-specific detail leaks through, making modification hard to debug
* While providing a CommonJS interface, the environment does not come with common NodeJS modules (e.g. fs, path, etc.), making reuse of code hard and creating new features harder.
* While providing a CommonJS interface, the environment does not come with common Node.js modules (e.g. fs, path, etc.), making reuse of code hard and creating new features harder.


It's therefore obvious to run build system with NodeJS instead. By substitute runtime dependency from XPCShell to NodeJS, we would get a stable and common runtime environment, and gain access to NPM packages.
In contrast, Node.js and the NPM packages runs on it can do things entirely simply in JavaScript. By substitute runtime dependency from XPCShell to Node.js, we would get a stable and common runtime environment, and gain access to NPM packages.


It's worthy to pointing out not building Gaia in XPCShell means we no longer considering making Firefox the sole dependency to build Gaia an eventual goal (i.e. develop Gaia with WebIDE & a "build Gaia" add-on and no extra dependency). Yet, considering the reward we decided using NodeJS is the better way moving forward.
It's worthy to pointing out not building Gaia in XPCShell means we no longer considering making Firefox the sole dependency to build Gaia an eventual goal (i.e. develop Gaia with WebIDE & a "build Gaia" add-on and no extra dependency). Yet, considering the reward we decided using Node.js is the better way moving forward.
 
= Profit from [http://nodejs.org/ Node.js] =
 
* Easier to catch up: Web developers are familiar with JavaScript, also [http://nodejs.org/ Node.js] is the most popular and acceptable for those who are familiar with client-side JavaScript. It could attract more contributors to involve and improve Firefox OS.
* Abundant tool: NPM is a well-known build-in package manager of node, such tool is commonly used for web developers for years. It will make your life easier that developers can install third-party libraries and prevent us to reinvent the wheel if there is no such library provided by XULRunner.
* Pure JS: It would be great if we build up a pure JS system. To re-implement Makefile into JS is a hard work without third-party library, NPM provides a bunch of packages that let us build our pure JS system more quickly, flexible and maintainable.


= Plan =
= Plan =
Line 36: Line 32:
== Milestone 1 ==
== Milestone 1 ==


Tackle each of the build system core functions (parallel build, webapp-manifests, preferences, webapp-optimize, webapp-zip... etc). All of them exist certain specific code only use for XPCShell. We would also like to finish the previous attempt to wrap NodeJS APIs into high-level Utils in (build/node-utils.js) (See also: {{Bug|955988}}). During this milestone, we will ensure the build system continue to work on both XPCShell setup and NodeJS. Once we complete all the work for the milestone, Gbn and Gbun should be green and set visible.
Tackle each of the build system core functions (parallel build, webapp-manifests, preferences, webapp-optimize, webapp-zip... etc). All of them exist certain specific code only use for XPCShell. We would also like to finish the previous attempt to wrap Node.js APIs into high-level Utils in (build/node-utils.js) (See also: {{Bug|955988}}). During this milestone, we will ensure the build system continue to work on both XPCShell setup and Node.js. Once we complete all the work for the milestone, Gbn and Gbun should be green and set visible.


* '''Deliverable:''' Complete the implementation of node build scripts that coexist with XPCShell and passes Gbn / Gbun tests
* '''Deliverable:''' Complete the implementation of node build scripts that coexist with XPCShell and passes Gbn / Gbun tests
Line 107: Line 103:




== Q & A ==
= Q & A =
 
== Which Gaia build task will continue to rely on b2g_sdk after Node.js transition? ==
 
b2g_sdk run on ...
 
* Building Gaia (including sub-commands)
* Running build integration test
* Running push to device
* '''Running marionette integration test (must)'''
 
All of the above can be replaced with node.js smoothly except marionette test. Since the B2G Desktop is the test target for these tests, it's impossible to remove from our dependency. However, it's not necessary to download a b2g_sdk package if someone do not execute marionette test.
 
== What of the effect on Gaia if build script run on node.js instead of b2g_sdk? ==
 
* Pros:
** Build Gaia without downloading b2g (but we still download b2g while launching marionette tests)
* Cons:
** Requiring Node.js as a build prerequisite -- User must make available <code>node</code> before building Gaia.
** We will need to download NPM packages and other NPM command-line tools
 
== Why are we giving up making Firefox as the only Gaia development dependency? Isn't that better and more inviting? ==
 
Weighting the benefits, we believe requiring Node.js won't frustrate many users to get started with hacking Gaia, since Node.js comes with installers for all platforms. Again, using Node.js means we can build up new features quickly by NPM and also speed up rewriting Makefile to pure JS.
 
On the other hand, try to rewrite Makefile from scratch in XULRunner is proven to be hard. In sum, we believe that NPM ecosystem will bring lots of benefits more than XULRunner.
 
== How do we ensure NPM package access to all parties, e.g. partner in China or Try servers ==
 
It should continue to work if we keep hosting our gaia-node-modules mirror to solve such situation. Replacing the make-shift gaia-node-modules mirror is not the scope of this work, but we are looking forward to find a way to do that.
 
== In M2, buildbot should pre-install proper command tools (node.js, gulp) if we decide to use gulp.js, right? ==
 
Buildbot has already installed node environment except gulp. We can pre-install gulp -g in buildbot, or download gulp while first time build then launch it by relative path.
 
== Would the new build system slower than we have right now? ==


# Which Gaia build task will continue to rely on b2g_sdk after NodeJS transition?
Although we believe that M1 & M3 won't effect obviously on performance, it probably happen on M2. So we should ensure that we take full advantage of multi-core systems to speed up our workflow in Node.js environment (with gulp).
#: b2g_sdk run on ...
#* Building Gaia (including sub-commands)
#* Running build integration test
#* Running push to device
#* '''Running marionette integration test (must)'''
#: All of the above can be replaced with node.js smoothly except marionette test. Since the B2G Desktop is the test target for these tests, it's impossible to remove from our dependency. However, it's not necessary to download a b2g_sdk package if someone do not execute marionette test.
# What of the effect on Gaia if build script run on node.js instead of b2g_sdk?
#: Pros:
#:* Build Gaia without downloading b2g (but we still download b2g while launching marionette tests)
#: Cons:
#:* We need to download NPM packages and pre-install node.js or other NPM command-line tools (probably be gulp?)
# WebIDE is trying to make developing Gaia more easily by clicking a button without any command line tricks. It triggers building process by invoking "make" currently. In XULRunner build system, after rewriting Makefile to pure JS module to take over it's job, WebIDE can build Gaia without any dependencies. (such as make and node)
#: For node build system, requiring [http://nodejs.org/ Node.js] may be a little bit inconvenient. However, it won't frustrate many users to get started with hacking Gaia since Node provides a great installer for all platforms. After [http://nodejs.org/ Node.js] installed, we can build up new features quickly by NPM and also speed up rewriting Makefile to pure JS. On the other hand, try to rewrite Makefile from scratch in XULRunner seems not so easy. I'm pretty sure and still believe that NPM ecosystem will bring lots of benefits more than XULRunner.
# Try-server or some countries have network issues when pulling packages from NPM.
#: It should work if we keep hosting our gaia-node-modules mirror to solve such situation.
# In stage 2, buildbot should pre-install proper command tools (node.js, gulp) if we decide to use gulp.js
#: Buildbot has already installed node environment except gulp. We can pre-install gulp -g in buildbot, or download gulp while first time build then launch it by relative path.
# We might take performance into account, the new build system shouldn't slower than the current.
#: Although I believe that stage 1 & 3 won't effect obviously on performance, it probably happen on stage 2. So we should ensure that we take full advantage of multi-core systems to speed up our workflow in node.js environment (in gulp.js).
Timdream
Confirmed users
478

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1053263"

Navigation menu