Gaia/Build/Refactoring Plan/First half of 2014

From MozillaWiki
Jump to: navigation, search

Since we found build scripts still isn’t easy to modify on bug 964216 and had bug 897352 for copying all apps into build_stage/ directory, we should consider refactoring build system to get clearer build process. There are something I found and how can we do for refactoring it.


Here is a simple version of diagram for our refactoring plan:


basically I would like to follow the idea on bug 897352 to copy all files in app into build_stage/ directory, and refactoring all build scripts to process files in build_stage/ instead of in app source directory and extract webapp-shared.js from webapp-zip.js to handle which files need to be picking from shared directory, webapp-zip.js is only used to zip files into profile.

Let’s show all details in below diagram and explain what should them do for each module:


Copy rule 
each app has a, if this file doesn’t exist we will use default way to copy files.
webapp-shared.js is used to pick files which is used in app from shared/ directory, this module will be extracted from webapp-zip.js
we load and use this module in webapp-zip.js for now, but it will be extracted to a xpcshell command and will be executed standalone.
fetch preload apps from internet if VARIANT_PATH exists and create directory for each app in build_stage.
each app should has their own makefile to generate and customize data, details are listed in next section.
optimize javascript, l10n files, html here, this module will be modified to optimize in build_stage directory instead of app source directory.
we have a lot of features in this modules for now, so we plan to move picking files in shared directory feature to webapp-shared.js and picking l10n files feature to multilocale.js, webapp-zip.js will be only used to compress file into profile directory.
additional-extension.js, preferences.js & settings.js 
nothing change.
extracted from webapp-manifest.js to generate webapps.json into profile directory.


(done, George) bug 968667 - create run-app-js-command  
create & export this macro to execute build script in app
bluetooth, clock, costcontrol, email, fl, fm, keyboard, music, ringtones, search, setringtone, video 
no change for stage 1.
(done, George) bug 968668 - browser 
need migrate some code from applications-data.js & webapp-manifests.js & webapp-optimize.js and it’s modified manifest so we need to build this app in build_stage.
(done, Yuren) bug 968669 - calendar
migrate some code from applications-data.js
(done, George) bug 968671 - camera
migrate some code form applications-data.js
(done, Yuren) bug 968675 - communications 
migrate some code from applications-data.js, webapp-manifests.js & webapp-zip.js, we need to build it in build_stage.
(done, George) bug 968680 - gallery 
migrate some code from applications-data.js
(done, Yuren) bug 968684 - homescreen 
migrate a lot of code from applications-data.js, webapp-zip
(done, Tim) bug 968688 - settings 
migrate some code from applications-data.js
(done, George) bug 968690 - sms 
migrate some code from applications-data.js
(done, George) bug 968697 - system 
migrate some code from applications-data.js, webapp-manifests.js & webapp-zip.js, we need to build it in build_stage
(done, Tim) bug 968702 - wallpaper 
migrate some code from webapp-zip.js and we need to build it in build_stage
(done, Tim) bug 968703 - wappush 
migrate some code from applications-data.js

Migration Plan

  1. (done, Greg) bug 968657 - Add |make watch| to watch files change and execute |APP=<app> make|, because we should always get files from build_stage/ for DEBUG=1 httpd.js.
  2. (done) Migrate build script for specific app from applications-data.js to app makefile.
  3. (done) bug 897352 - Add copy rule to copy app into build_stage/
  4. (done) bug 968661 - Extract new module webapp-shared.js from webapp-zip.js to copy files which are used in app to a specific directory
  5. (done) bug 968666 - multilocale.js should be a standalone build script which can be executed by macro |run-js-command|
  6. (done) bug 983573 - [Gaia] Refactoring webapp-manifest.js

we can do all action items 1, 2 in parallel and 4, 5 depend on 3.

Follow up bugs

  • bug 972228 - Verify default file content in non-distribution test
  • bug 969215 - execute |make| should only build specific apps we modified
  • (done, George) bug 971437 - Unify GAIA_DIR/customization and GAIA_DIR/build/test/resource/distribution_test
  • (done, Yuren) bug 972755 - use build_stage/preload-apps-data to store preload apps instead of GAIA_DISTRIBUTION_DIR/temp in variant.js
  • (invalid, Yuren) bug 973786 - Remove l10n/shared path handler in httpds/bootstrap.js and point app directory to build_stage/$APP