Gaia/Build/NewBuildSystem

Overview

After surveying, we decide to split build system into 2 steps: configure step and build step. For detailed information, please visit change proposal in dev-gaia Update on Gaia build changes: A configure step, and Make v.s. Gulp.

Our goal is to fasten incremental build and clobber build and easier to customize and manage new/current building step.

Plan

• Developer:
  • Ricky (MoCoTPE) (irc #gaia: rickychien)
  • George (MoCoTPE) (irc #gaia: gduan)

• Total expected schedule: 12 - 15 weeks

Milestone 1

Initial landing step. Since our new build system plan is around splitting a build process into two steps including configuring and building. This stage will focus on building up a configuration modules - configure.js and using GUN make as our build back-end to produce final target. It means configure.js module will run on XPCShell and require existed build modules at this stage in order to keep build system working properly. Finally, the new build system is going to be landed in code-base and replace current build system.

• Deliverable: A new Makefile target to trigger configure.js modules should be landed in Gaia code base. The configure.js can generate a Makefile and produce a same PROFILE_DIR as usual.
• Expected schedule: 4 weeks
• Breakdowns:
  • New config step: adding configure.js to generate strong dependency makefile and execute it.
  • New re-run step: in 2nd make, new build will check the change files and ENV to decide whether to regenerate makefile or not, and then execute it.
  • Refactoring webapp-shared.js and copyToStage: based on new build system architecture, we'll refactor webapp-shared module and copyToStage into config and execute task in order to save time when re-run make and check if any possible risk before Milestone 2 (refactoring all build modules and app/build.js).

Milestone 2

After initial landing step being done, we should abandon Makefile's PHONY targets as many as possible, so we could have a faster build by taking the advantage of Makefile's incremental build. The configure.js should be able to connect with different build back-end. In order to achieve this aim, we eventually come out a new phase - config.status for preparing the next step (replacing build back-end). The idea of config.status is charged with the task of tell back-end tool how to build the tree which includes scanning the build system definition and constructing a DAG data structure for representing task dependency and supporting parallel build natively. It means config.status will be generated by configure.js and output a build-file for back-end consumption (e.g. a `make` back-end will write a `Makefile`).

• Deliverable: Stabilization & Efficient incremental build & Implement config.status
• Expected schedule: 6 - 8 week
• Breakdowns:
  • Split all modules and app/build.js into small tasks. It means split into configuring & building steps.
  • Implement config.status which will introduce DAG structure to configure step.

Milestone 3

After surveying in milestone 3 of previous plan RefactoringToNode, we believe that the Gulp or streaming base build back-end could offer a better performance especially in clobber build. Therefore, in this step, a streaming base build system which will be implemented.

• Deliverable: A blazing fast JavaScript build system
• Expected schedule: week
• Breakdowns:
  • Implement a JavaScript build system as our build back-end tool instead of GNU make

Q & A

Why new build system is quicker than current one?

• clobber build
  • Current build: It manage process runner by app, which means the there would be idle processes if the last app takes longer time to complete.
  • New build: It manage process runner by file. The idle processes won't remain too long until all tasks complete.
• incremental build
  • Current build: If one file has changed, it will rebuild all the app (see below).

• • New build: If one file has changed, it will only trigger all the related tasks (see below).