MozillaWiki - User contributions [en]

Platform/2014-Q3-Goals

2014-07-07T21:49:25Z

Tglek: /* Release Engineering - Taras */

=== Platform ===
==== [[Platform/2014-Goals|2014 General Goals]] ====

=== GFX ===
Items marked [https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AnKFEBp1-VyqdFNfRlZmV0ExM0VvZGMxNThWX0d6LWc&usp=drive_web#gid=0 here] with release 33 and 34 are part of the Q3 landings.

=== Layout ===

=== Media ===

=== DOM ===
* {{ok|Mirror prototype of DOM objects through xray wrappers (peterv)}}
* {{ok|Remove nsDOMClassInfo.cpp}}
* {{ok|Make less-privileged non-Xrayable unwaived opaque from privileged code ({{bug|856067}})}}
* {{ok|Route all JSContext pushing through AutoJSAPI and Implement GetEntryGlobal ({{bug|951991}})}}

=== WebAPI ===

=== JS ===
* {{jok|837314}} ES6 classes
* {{jok|941796}} Generational GC on Firefox OS
* {{jok|650161}} Compacting GC to reduce memory usage
* {{jok|856533}} Escape analysis JIT optimizations
* {{jok|998392}} Use Latin1 strings to reduce memory usage
* {{jok|903519}} Allocate strings in GC nursery (for performance)
* {{jok|972710}} ARM64 JIT ''[stretch goal]''

=== Accessibility ===
* {{ok|}}e10s: proxy the common a11y API stuff like name, role, and states.
* {{ok|}}GAIA: Fix [https://bugzilla.mozilla.org/buglist.cgi?priority=--&f1=blocked&list_id=10674217&o1=anyexact&resolution=---&query_based_on=b2ga11y%20p%3D1&o2=substring&query_format=advanced&f2=status_whiteboard&v1=893789&component=Disability%20Access%20APIs&component=Gaia&component=Gaia%3A%3ABluetooth%20File%20Transfer&component=Gaia%3A%3ABrowser&component=Gaia%3A%3ABuild&component=Gaia%3A%3ACalendar&component=Gaia%3A%3ACamera&component=Gaia%3A%3AClock&component=Gaia%3A%3AContacts&component=Gaia%3A%3ACost%20Control&component=Gaia%3A%3ADialer&component=Gaia%3A%3AE-Mail&component=Gaia%3A%3AEverything.me&component=Gaia%3A%3AFirst%20Time%20Experience&component=Gaia%3A%3AFMRadio&component=Gaia%3A%3AGallery&component=Gaia%3A%3AGithubBot&component=Gaia%3A%3AHomescreen&component=Gaia%3A%3AKeyboard&component=Gaia%3A%3AL10n&component=Gaia%3A%3ALoop&component=Gaia%3A%3AMusic&component=Gaia%3A%3ANotes&component=Gaia%3A%3APDF%20Viewer&component=Gaia%3A%3APerformanceTest&component=Gaia%3A%3ARingtones&component=Gaia%3A%3ASearch&component=Gaia%3A%3ASettings&component=Gaia%3A%3ASMS&component=Gaia%3A%3ASystem&component=Gaia%3A%3ASystem%3A%3ABrowser%20Chrome&component=Gaia%3A%3ASystem%3A%3AInput%20Mgmt&component=Gaia%3A%3ASystem%3A%3ALockscreen&component=Gaia%3A%3ASystem%3A%3AWindow%20Mgmt&component=Gaia%3A%3ATestAgent&component=Gaia%3A%3AUI%20Tests&component=Gaia%3A%3AVideo&component=Gaia%3A%3AWallpaper&component=Gaia%3A%3AWappush&v2=b2ga11y%20p%3D1&product=Core&product=Firefox%20OS all Gaia P1 a11y bugs] (~30 at this time).
* {{ok|}}FFOS: {{Bug|1030465}} - Volume change should update the screen reader volume.
* {{ok|}}FFOS: {{Bug|1030466}} - Headphones screen reader volume is too low.
* {{ok|}}FFOS: {{Bug|1030468}} - VC rectangle needs to work with scaled content.
* {{ok|}}FFOS: {{Bug|1030470}} - Localization needs to work when switching locales in FxOS.

=== Perf ===

=== Networking ===

* {{ok|Ship Network Predictor ("Seer") on nightly, using HTTP cache instead of SQLite ({{bug|1009122}}) (hurley)}}
* {{ok|Ship current (hopefully final) IETF version of HTTP/2 preffed on in nightly (hurley)}}
* {{ok|HTTP/2 "alt-svc" support ({{bug|1003448}}) (mcmanus)}}
* {{ok|HTTP cache: determine if revalidation is needed w/o doing I/O ({{bug|983122}}) (sworkman) }}
* {{ok|Network up/down link detection on all platforms ({{bug|939318}}) (bagder) }}
* {{ok|Implement WebSocket compression extension ({{bug|792831}}) (michal) }}

=== Mobile ===

=== A*Team ===

For full list, see [[Auto-tools/Goals/2014Q3|A-Team Goals 2014Q3]]

'''B2G'''
* {{ok|}} Run a set of performance and correctness tests per-commit to b2g-inbound on Flame devices
* {{ok|}} Get gaia-integration tests running on device
* {{ok|}} Expand the FxOS Certification Suite with 1.4 support, test automation to prevent regressions, and investigation of support for non-phone devices
* {{ok|}} Green up B2G tests on TaskCluster (joint with RelEng)

'''Developer Productivity'''
* {{ok|}} Deploy ReviewBoard for developers to start using (joint with RelEng)

'''Performance'''
* {{ok|}} Deploy new Talos tests for tp5o_scroll, webgl, webrtc, and mainthread I/O
* {{ok|}} Get Datazilla alerts to beta mode (full parity with graph server alerts) with reduced noise
* {{ok|}} Get Eideticker running against Android again with increased frequency
* {{ok|}} Run B2G Eideticker against same branch/build combinations as our other on-device perf tests
* {{ok|}} Stand up a Games Benchmarking system for webaudio tests running against Firefox and Chrome

'''Treeherder'''
* {{ok|}} Deliver performance web service for ingesting and returning performance data
* {{ok|}} Deliver a UI for viewing Talos data

'''Sheriffing'''
* {{ok|}} Fully transition sheriffing from TBPL to Treeherder

'''General Automation'''
* {{ok|}} Create weekly reports that describe how many tests have been added/disabled/enabled per suite and platform
* {{ok|}} Move reftest to mozbase
* {{ok|}} Add command executors for Marionette for Java and Python

'''Bugzilla'''
* {{ok|}} Improve load time of related bugs; can decrease show_bug load times by up to 12%
* {{ok|}} Minify and concatenate JS files
* {{ok|}} Authoritative view for review history
* {{ok|}} Rewrite docs for REST API

'''Community'''
* {{ok|}} Create good_next_bugs (name can be adjusted) so once contributors are comfortable they can do more serious coding/problem solving on a project they are familiar with
* {{ok|}} Monthly review of mentored bugs and projects

=== Web Engineering ===
[https://etherpad.mozilla.org/webeng-q32014 full list - high level summary to come]

=== SUMO and Input ===
* SUMO: Launch new offline SUMO app [Get Firefox on a Growth Trajectory]
* SUMO: Develop new Community Hub to a level where it can replace legacy Karma app [Enable Communities with Impact]
* SUMO: Research and prototype new experimental features (e.g. geotargeting, Instant Search, Search Suggestions) [Get Firefox on a Growth Trajectory]
* Input: Improve documentation and install to lower the bar for contribution [Enable Communities with Impact]
* Input: Ship Firefox Desktop product dashboard [Get Firefox on a Growth Trajectory]
* Input: (STRETCH) Ship Firefox OS product dashboard [Scaling FirefoxOS]

=== Release Engineering - Laura ===
* Enable capacity expansion for bare-metal releng OS X build and test slaves via hardware provided by third-party datacenters (Get Firefox on a Trajectory of Growth)
* Simplify release engineering mobile hardware infrastructure (Get Firefox on a Trajectory of Growth)
* Automate developer access to continuous integration resources to expedite debugging and standing up new job variants. (Enable Communities With Impact)
* Help enable video operability on the web by providing continuous integration for Cisco Open H.264 builds. (Get Firefox on a Trajectory of Growth)

=== Release Engineering - Taras ===
* Replace aging Firefox update service with a scalable, modern solution. (Get Firefox on a Trajectory of Growth)
* Simplify developer workflow by automating patch landing and uplift. (Enable Communities With Impact)
* Local HTTP caches in each region/datacenter: sccache, ftp.mozilla.org, pvtbuilds
* EBS < 3% of AWS bill(vs 30% in Q2)
* 80% of b2g jobs turned off in buildbut, running on taskcluster
* Turn telemetry is into a general purpose s3 ingester and analysis tool

=== Release Engineering Operations ===
* Build/Test System Performance Enhancements
** {{ok| Unify releng platform architecture, using common tools and best practices, to decrease complexity and enable smoother developer engagement {{bug|1026109}} and {{bug|1026110}} [Get Firefox on a Trajectory of Growth, Enable Communities With Impact] }}
** {{ok| Improve communication and response time for releng network flow requests {{bug|1026112}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Improve security for all releng windows infrastructure {{bug|893716}} [Get Firefox on a Trajectory of Growth] }}
* Build/Test System Self-Serve Re-architecture
** {{prev| Design a private cloud deployment architecture for bare metal and produce a POC that supports Ubuntu 12.04 test machines. {{bug|963165}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox linux test slaves (build slaves as a stretch goal) {{bug|1026687}} (depends on completion of previous goal) [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox windows test slaves. {{bug|967064}} [Get Firefox on a Growth Trajectory] }}

=== Developer Services ===
* {{new|(with B-team) roll out phase 2 of new review tooling}} [Get Firefox on a Growth Trajectory / Enable communities with impact]
* {{new|Upgrade existing Mercurial infrastructure to support more rapid, safe, and coordinated deployment of updates.}} [Get Firefox on a Growth Trajectory / Enable communities with impact]

[https://etherpad.mozilla.org/devservices-Q32014 Full List]

=== Security & Privacy Engineering ===
More details here: [[SecurityEngineering/2014/Q3Goals]]

'''Content Security'''
* {{new|Gecko Security Hooks: Finish code and debugging for New Channel API, start getting reviews}} (dri=tanvi)
* {{new|Gecko Security Hooks: Create plan for addon compatibility}} (dri=tanvi)
* {{new|CSP: Remove old JS implementation from mozilla-central}} (dri=sstamm)
* {{new|Evangelism: Security Open Mic presentation + blog post about new CSP implementation, maybe again as brown bag.}} (dri=sstamm)
* {{new|''[stretch goal]'' CSP: Fix majority of CSP 1.1 compatibility bugs}} (dri=ckerschb)

'''Tracking Protection'''
* {{new|Referer: Finish implementation of <meta> referrer control with volunteer help}} (dri=sstamm)
* {{new|Land first implementation of protection in Fx 33/34 off by default.}} (dri=mmc)

'''Communications Security'''
* {{prev|SSL Error Reporting finish first implementation of ssl error reporting feature.}} (dri=grobinson)
* {{new| HPKP - implement pinning http header}} (dri=cviecco)
* {{new| Update roadmap for Cert Revocation improvements}} (dri=rbarnes)
* {{new| Create a mechanism to provision phones with an alternate cert}} (dri=mgoodwin)
* {{new| Add measurement/enforcement of compliance with CABF Baseline Requirements}} (dri=keeler)
* {{new| Create a tool for testing CA certificate compliance and EV-readiness}} (dri=keeler)
* {{new| Add support for key wrap/unwrap and ECC in WebCrypto}} (dri=rbarnes)
* {{new| ''[stretch goal]'' Get CA Program data into one database}} (dri=kathleen)

=== Firefox and Platform Security ===

* [https://mana.mozilla.org/wiki/display/~gkwong@mozilla.com/Marifuzz Marifuzz ] fuzzer ported to and running on Flame devices.
* Update ASan and LSan work for DOMFuzzer
* Update "Bounty Stars" document with issues found by external reporters and updating DOMFuzzer to reflect these results.
* Get Clang on RelEng ready for official OS X ASan builds.
* Initial work to move CoreFuzz towards running in cloud environments.
* WebCrypto API fuzzing using Dharma fuzzer.
* Port a portion of WebRTC fuzzing from Frambois fuzzer to Dharma fuzzer.
* Peach: Improving and porting Peach 2 to Python 3.
* Public Mozilla Security Github work: Moving of fuzzing tools from Fuzzing Hg to GitHub, including work to separate harnesses from testcase generation tools.

=== Games Program ===

=== Release Management ===
For full list, see [[Release_Management/Goals/2014Q3#Release_Management_General|Release Management 2014Q3 Goals]].
* Create and document process for Desktop/Mobile feature fast tracking
* Determine future of ESR and how to manage this channel
* Continue desktop throttling experiment with intention of reducing throttled time while maintaining existing level of feedback
* Improve release notes with revamped template for all products
* Create B2G release model proposals and gather feedback for potential changes
* Figure out what to do with B2G Security Releases

Platform/2014-Q3-Goals

2014-07-07T21:41:12Z

Tglek:

=== Platform ===
==== [[Platform/2014-Goals|2014 General Goals]] ====

=== GFX ===
Items marked [https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AnKFEBp1-VyqdFNfRlZmV0ExM0VvZGMxNThWX0d6LWc&usp=drive_web#gid=0 here] with release 33 and 34 are part of the Q3 landings.

=== Layout ===

=== Media ===

=== DOM ===
* {{ok|Mirror prototype of DOM objects through xray wrappers (peterv)}}
* {{ok|Remove nsDOMClassInfo.cpp}}
* {{ok|Make less-privileged non-Xrayable unwaived opaque from privileged code ({{bug|856067}})}}
* {{ok|Route all JSContext pushing through AutoJSAPI and Implement GetEntryGlobal ({{bug|951991}})}}

=== WebAPI ===

=== JS ===
* {{jok|837314}} ES6 classes
* {{jok|941796}} Generational GC on Firefox OS
* {{jok|650161}} Compacting GC to reduce memory usage
* {{jok|856533}} Escape analysis JIT optimizations
* {{jok|998392}} Use Latin1 strings to reduce memory usage
* {{jok|903519}} Allocate strings in GC nursery (for performance)
* {{jok|972710}} ARM64 JIT ''[stretch goal]''

=== Accessibility ===
* {{ok|}}e10s: proxy the common a11y API stuff like name, role, and states.
* {{ok|}}GAIA: Fix [https://bugzilla.mozilla.org/buglist.cgi?priority=--&f1=blocked&list_id=10674217&o1=anyexact&resolution=---&query_based_on=b2ga11y%20p%3D1&o2=substring&query_format=advanced&f2=status_whiteboard&v1=893789&component=Disability%20Access%20APIs&component=Gaia&component=Gaia%3A%3ABluetooth%20File%20Transfer&component=Gaia%3A%3ABrowser&component=Gaia%3A%3ABuild&component=Gaia%3A%3ACalendar&component=Gaia%3A%3ACamera&component=Gaia%3A%3AClock&component=Gaia%3A%3AContacts&component=Gaia%3A%3ACost%20Control&component=Gaia%3A%3ADialer&component=Gaia%3A%3AE-Mail&component=Gaia%3A%3AEverything.me&component=Gaia%3A%3AFirst%20Time%20Experience&component=Gaia%3A%3AFMRadio&component=Gaia%3A%3AGallery&component=Gaia%3A%3AGithubBot&component=Gaia%3A%3AHomescreen&component=Gaia%3A%3AKeyboard&component=Gaia%3A%3AL10n&component=Gaia%3A%3ALoop&component=Gaia%3A%3AMusic&component=Gaia%3A%3ANotes&component=Gaia%3A%3APDF%20Viewer&component=Gaia%3A%3APerformanceTest&component=Gaia%3A%3ARingtones&component=Gaia%3A%3ASearch&component=Gaia%3A%3ASettings&component=Gaia%3A%3ASMS&component=Gaia%3A%3ASystem&component=Gaia%3A%3ASystem%3A%3ABrowser%20Chrome&component=Gaia%3A%3ASystem%3A%3AInput%20Mgmt&component=Gaia%3A%3ASystem%3A%3ALockscreen&component=Gaia%3A%3ASystem%3A%3AWindow%20Mgmt&component=Gaia%3A%3ATestAgent&component=Gaia%3A%3AUI%20Tests&component=Gaia%3A%3AVideo&component=Gaia%3A%3AWallpaper&component=Gaia%3A%3AWappush&v2=b2ga11y%20p%3D1&product=Core&product=Firefox%20OS all Gaia P1 a11y bugs] (~30 at this time).
* {{ok|}}FFOS: {{Bug|1030465}} - Volume change should update the screen reader volume.
* {{ok|}}FFOS: {{Bug|1030466}} - Headphones screen reader volume is too low.
* {{ok|}}FFOS: {{Bug|1030468}} - VC rectangle needs to work with scaled content.
* {{ok|}}FFOS: {{Bug|1030470}} - Localization needs to work when switching locales in FxOS.

=== Perf ===

=== Networking ===

* {{ok|Ship Network Predictor ("Seer") on nightly, using HTTP cache instead of SQLite ({{bug|1009122}}) (hurley)}}
* {{ok|Ship current (hopefully final) IETF version of HTTP/2 preffed on in nightly (hurley)}}
* {{ok|HTTP/2 "alt-svc" support ({{bug|1003448}}) (mcmanus)}}
* {{ok|HTTP cache: determine if revalidation is needed w/o doing I/O ({{bug|983122}}) (sworkman) }}
* {{ok|Network up/down link detection on all platforms ({{bug|939318}}) (bagder) }}
* {{ok|Implement WebSocket compression extension ({{bug|792831}}) (michal) }}

=== Mobile ===

=== A*Team ===

For full list, see [[Auto-tools/Goals/2014Q3|A-Team Goals 2014Q3]]

'''B2G'''
* {{ok|}} Run a set of performance and correctness tests per-commit to b2g-inbound on Flame devices
* {{ok|}} Get gaia-integration tests running on device
* {{ok|}} Expand the FxOS Certification Suite with 1.4 support, test automation to prevent regressions, and investigation of support for non-phone devices
* {{ok|}} Green up B2G tests on TaskCluster (joint with RelEng)

'''Developer Productivity'''
* {{ok|}} Deploy ReviewBoard for developers to start using (joint with RelEng)

'''Performance'''
* {{ok|}} Deploy new Talos tests for tp5o_scroll, webgl, webrtc, and mainthread I/O
* {{ok|}} Get Datazilla alerts to beta mode (full parity with graph server alerts) with reduced noise
* {{ok|}} Get Eideticker running against Android again with increased frequency
* {{ok|}} Run B2G Eideticker against same branch/build combinations as our other on-device perf tests
* {{ok|}} Stand up a Games Benchmarking system for webaudio tests running against Firefox and Chrome

'''Treeherder'''
* {{ok|}} Deliver performance web service for ingesting and returning performance data
* {{ok|}} Deliver a UI for viewing Talos data

'''Sheriffing'''
* {{ok|}} Fully transition sheriffing from TBPL to Treeherder

'''General Automation'''
* {{ok|}} Create weekly reports that describe how many tests have been added/disabled/enabled per suite and platform
* {{ok|}} Move reftest to mozbase
* {{ok|}} Add command executors for Marionette for Java and Python

'''Bugzilla'''
* {{ok|}} Improve load time of related bugs; can decrease show_bug load times by up to 12%
* {{ok|}} Minify and concatenate JS files
* {{ok|}} Authoritative view for review history
* {{ok|}} Rewrite docs for REST API

'''Community'''
* {{ok|}} Create good_next_bugs (name can be adjusted) so once contributors are comfortable they can do more serious coding/problem solving on a project they are familiar with
* {{ok|}} Monthly review of mentored bugs and projects

=== Web Engineering ===
[https://etherpad.mozilla.org/webeng-q32014 full list - high level summary to come]

=== SUMO and Input ===
* SUMO: Launch new offline SUMO app [Get Firefox on a Growth Trajectory]
* SUMO: Develop new Community Hub to a level where it can replace legacy Karma app [Enable Communities with Impact]
* SUMO: Research and prototype new experimental features (e.g. geotargeting, Instant Search, Search Suggestions) [Get Firefox on a Growth Trajectory]
* Input: Improve documentation and install to lower the bar for contribution [Enable Communities with Impact]
* Input: Ship Firefox Desktop product dashboard [Get Firefox on a Growth Trajectory]
* Input: (STRETCH) Ship Firefox OS product dashboard [Scaling FirefoxOS]

=== Release Engineering - Laura ===
* Enable capacity expansion for bare-metal releng OS X build and test slaves via hardware provided by third-party datacenters (Get Firefox on a Trajectory of Growth)
* Simplify release engineering mobile hardware infrastructure (Get Firefox on a Trajectory of Growth)
* Automate developer access to continuous integration resources to expedite debugging and standing up new job variants. (Enable Communities With Impact)
* Help enable video operability on the web by providing continuous integration for Cisco Open H.264 builds. (Get Firefox on a Trajectory of Growth)

=== Release Engineering - Taras ===
* Replace aging Firefox update service with a scalable, modern solution. (Get Firefox on a Trajectory of Growth)
* Simplify developer workflow by automating patch landing and uplift. (Enable Communities With Impact)
* Local HTTP caches in each region/datacenter: sccache, ftp.mozilla.org, pvtbuilds
* EBS < 3% of AWS bill(vs 30% in Q2)
* 80% of b2g jobs turned off in buildbut, running on taskcluster

=== Release Engineering Operations ===
* Build/Test System Performance Enhancements
** {{ok| Unify releng platform architecture, using common tools and best practices, to decrease complexity and enable smoother developer engagement {{bug|1026109}} and {{bug|1026110}} [Get Firefox on a Trajectory of Growth, Enable Communities With Impact] }}
** {{ok| Improve communication and response time for releng network flow requests {{bug|1026112}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Improve security for all releng windows infrastructure {{bug|893716}} [Get Firefox on a Trajectory of Growth] }}
* Build/Test System Self-Serve Re-architecture
** {{prev| Design a private cloud deployment architecture for bare metal and produce a POC that supports Ubuntu 12.04 test machines. {{bug|963165}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox linux test slaves (build slaves as a stretch goal) {{bug|1026687}} (depends on completion of previous goal) [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox windows test slaves. {{bug|967064}} [Get Firefox on a Growth Trajectory] }}

=== Developer Services ===
* {{new|(with B-team) roll out phase 2 of new review tooling}} [Get Firefox on a Growth Trajectory / Enable communities with impact]
* {{new|Upgrade existing Mercurial infrastructure to support more rapid, safe, and coordinated deployment of updates.}} [Get Firefox on a Growth Trajectory / Enable communities with impact]

[https://etherpad.mozilla.org/devservices-Q32014 Full List]

=== Security & Privacy Engineering ===
More details here: [[SecurityEngineering/2014/Q3Goals]]

'''Content Security'''
* {{new|Gecko Security Hooks: Finish code and debugging for New Channel API, start getting reviews}} (dri=tanvi)
* {{new|Gecko Security Hooks: Create plan for addon compatibility}} (dri=tanvi)
* {{new|CSP: Remove old JS implementation from mozilla-central}} (dri=sstamm)
* {{new|Evangelism: Security Open Mic presentation + blog post about new CSP implementation, maybe again as brown bag.}} (dri=sstamm)
* {{new|''[stretch goal]'' CSP: Fix majority of CSP 1.1 compatibility bugs}} (dri=ckerschb)

'''Tracking Protection'''
* {{new|Referer: Finish implementation of <meta> referrer control with volunteer help}} (dri=sstamm)
* {{new|Land first implementation of protection in Fx 33/34 off by default.}} (dri=mmc)

'''Communications Security'''
* {{prev|SSL Error Reporting finish first implementation of ssl error reporting feature.}} (dri=grobinson)
* {{new| HPKP - implement pinning http header}} (dri=cviecco)
* {{new| Update roadmap for Cert Revocation improvements}} (dri=rbarnes)
* {{new| Create a mechanism to provision phones with an alternate cert}} (dri=mgoodwin)
* {{new| Add measurement/enforcement of compliance with CABF Baseline Requirements}} (dri=keeler)
* {{new| Create a tool for testing CA certificate compliance and EV-readiness}} (dri=keeler)
* {{new| Add support for key wrap/unwrap and ECC in WebCrypto}} (dri=rbarnes)
* {{new| ''[stretch goal]'' Get CA Program data into one database}} (dri=kathleen)

=== Firefox and Platform Security ===

* [https://mana.mozilla.org/wiki/display/~gkwong@mozilla.com/Marifuzz Marifuzz ] fuzzer ported to and running on Flame devices.
* Update ASan and LSan work for DOMFuzzer
* Update "Bounty Stars" document with issues found by external reporters and updating DOMFuzzer to reflect these results.
* Get Clang on RelEng ready for official OS X ASan builds.
* Initial work to move CoreFuzz towards running in cloud environments.
* WebCrypto API fuzzing using Dharma fuzzer.
* Port a portion of WebRTC fuzzing from Frambois fuzzer to Dharma fuzzer.
* Peach: Improving and porting Peach 2 to Python 3.
* Public Mozilla Security Github work: Moving of fuzzing tools from Fuzzing Hg to GitHub, including work to separate harnesses from testcase generation tools.

=== Games Program ===

=== Release Management ===
For full list, see [[Release_Management/Goals/2014Q3#Release_Management_General|Release Management 2014Q3 Goals]].
* Create and document process for Desktop/Mobile feature fast tracking
* Determine future of ESR and how to manage this channel
* Continue desktop throttling experiment with intention of reducing throttled time while maintaining existing level of feedback
* Improve release notes with revamped template for all products
* Create B2G release model proposals and gather feedback for potential changes
* Figure out what to do with B2G Security Releases

Platform/2014-Q3-Goals

2014-07-07T21:32:02Z

Tglek: /* Release Engineering */

=== Platform ===
==== [[Platform/2014-Goals|2014 General Goals]] ====

=== GFX ===
Items marked [https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AnKFEBp1-VyqdFNfRlZmV0ExM0VvZGMxNThWX0d6LWc&usp=drive_web#gid=0 here] with release 33 and 34 are part of the Q3 landings.

=== Layout ===

=== Media ===

=== DOM ===
* {{ok|Mirror prototype of DOM objects through xray wrappers (peterv)}}
* {{ok|Remove nsDOMClassInfo.cpp}}
* {{ok|Make less-privileged non-Xrayable unwaived opaque from privileged code ({{bug|856067}})}}
* {{ok|Route all JSContext pushing through AutoJSAPI and Implement GetEntryGlobal ({{bug|951991}})}}

=== WebAPI ===

=== JS ===
* {{jok|837314}} ES6 classes
* {{jok|941796}} Generational GC on Firefox OS
* {{jok|650161}} Compacting GC to reduce memory usage
* {{jok|856533}} Escape analysis JIT optimizations
* {{jok|998392}} Use Latin1 strings to reduce memory usage
* {{jok|903519}} Allocate strings in GC nursery (for performance)
* {{jok|972710}} ARM64 JIT ''[stretch goal]''

=== Accessibility ===
* {{ok|}}e10s: proxy the common a11y API stuff like name, role, and states.
* {{ok|}}GAIA: Fix [https://bugzilla.mozilla.org/buglist.cgi?priority=--&f1=blocked&list_id=10674217&o1=anyexact&resolution=---&query_based_on=b2ga11y%20p%3D1&o2=substring&query_format=advanced&f2=status_whiteboard&v1=893789&component=Disability%20Access%20APIs&component=Gaia&component=Gaia%3A%3ABluetooth%20File%20Transfer&component=Gaia%3A%3ABrowser&component=Gaia%3A%3ABuild&component=Gaia%3A%3ACalendar&component=Gaia%3A%3ACamera&component=Gaia%3A%3AClock&component=Gaia%3A%3AContacts&component=Gaia%3A%3ACost%20Control&component=Gaia%3A%3ADialer&component=Gaia%3A%3AE-Mail&component=Gaia%3A%3AEverything.me&component=Gaia%3A%3AFirst%20Time%20Experience&component=Gaia%3A%3AFMRadio&component=Gaia%3A%3AGallery&component=Gaia%3A%3AGithubBot&component=Gaia%3A%3AHomescreen&component=Gaia%3A%3AKeyboard&component=Gaia%3A%3AL10n&component=Gaia%3A%3ALoop&component=Gaia%3A%3AMusic&component=Gaia%3A%3ANotes&component=Gaia%3A%3APDF%20Viewer&component=Gaia%3A%3APerformanceTest&component=Gaia%3A%3ARingtones&component=Gaia%3A%3ASearch&component=Gaia%3A%3ASettings&component=Gaia%3A%3ASMS&component=Gaia%3A%3ASystem&component=Gaia%3A%3ASystem%3A%3ABrowser%20Chrome&component=Gaia%3A%3ASystem%3A%3AInput%20Mgmt&component=Gaia%3A%3ASystem%3A%3ALockscreen&component=Gaia%3A%3ASystem%3A%3AWindow%20Mgmt&component=Gaia%3A%3ATestAgent&component=Gaia%3A%3AUI%20Tests&component=Gaia%3A%3AVideo&component=Gaia%3A%3AWallpaper&component=Gaia%3A%3AWappush&v2=b2ga11y%20p%3D1&product=Core&product=Firefox%20OS all Gaia P1 a11y bugs] (~30 at this time).
* {{ok|}}FFOS: {{Bug|1030465}} - Volume change should update the screen reader volume.
* {{ok|}}FFOS: {{Bug|1030466}} - Headphones screen reader volume is too low.
* {{ok|}}FFOS: {{Bug|1030468}} - VC rectangle needs to work with scaled content.
* {{ok|}}FFOS: {{Bug|1030470}} - Localization needs to work when switching locales in FxOS.

=== Perf ===

=== Networking ===

* {{ok|Ship Network Predictor ("Seer") on nightly, using HTTP cache instead of SQLite ({{bug|1009122}}) (hurley)}}
* {{ok|Ship current (hopefully final) IETF version of HTTP/2 preffed on in nightly (hurley)}}
* {{ok|HTTP/2 "alt-svc" support ({{bug|1003448}}) (mcmanus)}}
* {{ok|HTTP cache: determine if revalidation is needed w/o doing I/O ({{bug|983122}}) (sworkman) }}
* {{ok|Network up/down link detection on all platforms ({{bug|939318}}) (bagder) }}
* {{ok|Implement WebSocket compression extension ({{bug|792831}}) (michal) }}

=== Mobile ===

=== A*Team ===

For full list, see [[Auto-tools/Goals/2014Q3|A-Team Goals 2014Q3]]

'''B2G'''
* {{ok|}} Run a set of performance and correctness tests per-commit to b2g-inbound on Flame devices
* {{ok|}} Get gaia-integration tests running on device
* {{ok|}} Expand the FxOS Certification Suite with 1.4 support, test automation to prevent regressions, and investigation of support for non-phone devices
* {{ok|}} Green up B2G tests on TaskCluster (joint with RelEng)

'''Developer Productivity'''
* {{ok|}} Deploy ReviewBoard for developers to start using (joint with RelEng)

'''Performance'''
* {{ok|}} Deploy new Talos tests for tp5o_scroll, webgl, webrtc, and mainthread I/O
* {{ok|}} Get Datazilla alerts to beta mode (full parity with graph server alerts) with reduced noise
* {{ok|}} Get Eideticker running against Android again with increased frequency
* {{ok|}} Run B2G Eideticker against same branch/build combinations as our other on-device perf tests
* {{ok|}} Stand up a Games Benchmarking system for webaudio tests running against Firefox and Chrome

'''Treeherder'''
* {{ok|}} Deliver performance web service for ingesting and returning performance data
* {{ok|}} Deliver a UI for viewing Talos data

'''Sheriffing'''
* {{ok|}} Fully transition sheriffing from TBPL to Treeherder

'''General Automation'''
* {{ok|}} Create weekly reports that describe how many tests have been added/disabled/enabled per suite and platform
* {{ok|}} Move reftest to mozbase
* {{ok|}} Add command executors for Marionette for Java and Python

'''Bugzilla'''
* {{ok|}} Improve load time of related bugs; can decrease show_bug load times by up to 12%
* {{ok|}} Minify and concatenate JS files
* {{ok|}} Authoritative view for review history
* {{ok|}} Rewrite docs for REST API

'''Community'''
* {{ok|}} Create good_next_bugs (name can be adjusted) so once contributors are comfortable they can do more serious coding/problem solving on a project they are familiar with
* {{ok|}} Monthly review of mentored bugs and projects

=== Web Engineering ===
[https://etherpad.mozilla.org/webeng-q32014 full list - high level summary to come]

=== SUMO and Input ===
* SUMO: Launch new offline SUMO app [Get Firefox on a Growth Trajectory]
* SUMO: Develop new Community Hub to a level where it can replace legacy Karma app [Enable Communities with Impact]
* SUMO: Research and prototype new experimental features (e.g. geotargeting, Instant Search, Search Suggestions) [Get Firefox on a Growth Trajectory]
* Input: Improve documentation and install to lower the bar for contribution [Enable Communities with Impact]
* Input: Ship Firefox Desktop product dashboard [Get Firefox on a Growth Trajectory]
* Input: (STRETCH) Ship Firefox OS product dashboard [Scaling FirefoxOS]

=== Release Engineering - Laura ===
* Enable capacity expansion for bare-metal releng OS X build and test slaves via hardware provided by third-party datacenters (Get Firefox on a Trajectory of Growth)
* Simplify release engineering mobile hardware infrastructure (Get Firefox on a Trajectory of Growth)
* Automate developer access to continuous integration resources to expedite debugging and standing up new job variants. (Enable Communities With Impact)
* Help enable video operability on the web by providing continuous integration for Cisco Open H.264 builds. (Get Firefox on a Trajectory of Growth)
* Replace aging Firefox update service with a scaleable, modern solution. (Get Firefox on a Trajectory of Growth)
* Simplify developer workflow by automating patch landing and uplift. (Enable Communities With Impact)

=== Release Engineering Operations ===
* Build/Test System Performance Enhancements
** {{ok| Unify releng platform architecture, using common tools and best practices, to decrease complexity and enable smoother developer engagement {{bug|1026109}} and {{bug|1026110}} [Get Firefox on a Trajectory of Growth, Enable Communities With Impact] }}
** {{ok| Improve communication and response time for releng network flow requests {{bug|1026112}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Improve security for all releng windows infrastructure {{bug|893716}} [Get Firefox on a Trajectory of Growth] }}
* Build/Test System Self-Serve Re-architecture
** {{prev| Design a private cloud deployment architecture for bare metal and produce a POC that supports Ubuntu 12.04 test machines. {{bug|963165}} [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox linux test slaves (build slaves as a stretch goal) {{bug|1026687}} (depends on completion of previous goal) [Get Firefox on a Trajectory of Growth, Scale Firefox OS] }}
** {{ok| Create self-service capability for releng hardware Firefox windows test slaves. {{bug|967064}} [Get Firefox on a Growth Trajectory] }}

=== Developer Services ===
* {{new|(with B-team) roll out phase 2 of new review tooling}} [Get Firefox on a Growth Trajectory / Enable communities with impact]
* {{new|Upgrade existing Mercurial infrastructure to support more rapid, safe, and coordinated deployment of updates.}} [Get Firefox on a Growth Trajectory / Enable communities with impact]

[https://etherpad.mozilla.org/devservices-Q32014 Full List]

=== Security & Privacy Engineering ===
More details here: [[SecurityEngineering/2014/Q3Goals]]

'''Content Security'''
* {{new|Gecko Security Hooks: Finish code and debugging for New Channel API, start getting reviews}} (dri=tanvi)
* {{new|Gecko Security Hooks: Create plan for addon compatibility}} (dri=tanvi)
* {{new|CSP: Remove old JS implementation from mozilla-central}} (dri=sstamm)
* {{new|Evangelism: Security Open Mic presentation + blog post about new CSP implementation, maybe again as brown bag.}} (dri=sstamm)
* {{new|''[stretch goal]'' CSP: Fix majority of CSP 1.1 compatibility bugs}} (dri=ckerschb)

'''Tracking Protection'''
* {{new|Referer: Finish implementation of <meta> referrer control with volunteer help}} (dri=sstamm)
* {{new|Land first implementation of protection in Fx 33/34 off by default.}} (dri=mmc)

'''Communications Security'''
* {{prev|SSL Error Reporting finish first implementation of ssl error reporting feature.}} (dri=grobinson)
* {{new| HPKP - implement pinning http header}} (dri=cviecco)
* {{new| Update roadmap for Cert Revocation improvements}} (dri=rbarnes)
* {{new| Create a mechanism to provision phones with an alternate cert}} (dri=mgoodwin)
* {{new| Add measurement/enforcement of compliance with CABF Baseline Requirements}} (dri=keeler)
* {{new| Create a tool for testing CA certificate compliance and EV-readiness}} (dri=keeler)
* {{new| Add support for key wrap/unwrap and ECC in WebCrypto}} (dri=rbarnes)
* {{new| ''[stretch goal]'' Get CA Program data into one database}} (dri=kathleen)

=== Firefox and Platform Security ===

* [https://mana.mozilla.org/wiki/display/~gkwong@mozilla.com/Marifuzz Marifuzz ] fuzzer ported to and running on Flame devices.
* Update ASan and LSan work for DOMFuzzer
* Update "Bounty Stars" document with issues found by external reporters and updating DOMFuzzer to reflect these results.
* Get Clang on RelEng ready for official OS X ASan builds.
* Initial work to move CoreFuzz towards running in cloud environments.
* WebCrypto API fuzzing using Dharma fuzzer.
* Port a portion of WebRTC fuzzing from Frambois fuzzer to Dharma fuzzer.
* Peach: Improving and porting Peach 2 to Python 3.
* Public Mozilla Security Github work: Moving of fuzzing tools from Fuzzing Hg to GitHub, including work to separate harnesses from testcase generation tools.

=== Games Program ===

=== Release Management ===
For full list, see [[Release_Management/Goals/2014Q3#Release_Management_General|Release Management 2014Q3 Goals]].
* Create and document process for Desktop/Mobile feature fast tracking
* Determine future of ESR and how to manage this channel
* Continue desktop throttling experiment with intention of reducing throttled time while maintaining existing level of feedback
* Improve release notes with revamped template for all products
* Create B2G release model proposals and gather feedback for potential changes
* Figure out what to do with B2G Security Releases

Mobile/Janus

2014-07-07T17:04:22Z

Tglek: /* Goals */

Janus is a project to find and evaluate ways to enable more efficient mobile browsing.

== Goals ==
The goal of Janus is to verify whether we can enhance the mobile browsing experience by achieving the following improvements in a practical way:
* Reduce radio time
* Reduce page load times
* Reduce bandwidth requirements
* '''Increase user privacy'''
* Increase responsiveness for slow sites

Ways we hope to accomplish some of these things:
* Recompress images to be smaller without noticeable visual impact
* Compress text (js, html) streams (gzip, bzip2, lzma?)
* Caching
* Reduce HTTPS round trips
* Move safe browsing support to the proxy
* Link prefetching

Other crazy stuff we could do:
* Man-in-the-middle HTTPS traffic only for images (so they get recompressed) with user consent
* Convert animated GIF to H264/WebM/whatever. gfycat.com does this now with apparently good results
* Pre-Shumwayize Flash content
* Automatic readability mode for some sites
* Ad blocking
** Only obnoxious ads?
** Replace with our own?
** Only block ones that track?
* Add support for adaptive streaming (MPEG-DASH) for servers that don't support it. Transcode to lower bitrates.
** Proxy HLS sites as MPEG-DASH
* Opera Mini-like pre-rendering of pages

Taras crazy stuff:
* http://zsync.moria.org.uk/ content that changes a little bit, but frequently
* Can also do this for web cut/n/paste. Server could insist on serving zsync summaries of common files(eg css libs, js libs like jquery) and client would do a distance search locally to see if it has anything similar and then only do range requests needed to construct the new version.

<code>
Currently we send if-modified-since:
We could add header supports-zsync:

This would make server reply with a .zsync which the client would then use to submit range: requests.
</code>
* asset bundling/inlining for cold loads
** server can determine when client got nothing in cache(eg no if-modified-since), do spdy-push from a preset profile
** server could spdy-push partial files(eg only css that gets used) + zsync of rest of contents so client can request missing pieces for other pages on the website(this can hugely reduce transfer sizes)...If i remember correctly from https://bugzilla.mozilla.org/show_bug.cgi?id=834865 70%+ of css is leftover cruft...Note this might be easier to do having sites be aware of this and bundling a library that supports having css/js broken up into multiple files & supports waiting on missing pieces

== Background ==

=== Transport Layer Security (TLS) ===
==== TLS handshake ====
<code>
'''Time''' [ms] '''Client''' '''Server'''
30 SYN -->
60 <-- SYN ACK
90 ACK -->
120 <-- Certificate
150 ChangeChiperSpec -->
180 <-- ChangeCipherSpec
210 Data -->
240 <-- Data
</code>

=== 3G/4G ===
==== LTE Mobile Network ====
<code>
'''Latency''' [ms] '''Type''' '''Device''' '''RAN''' '''Core Network''' '''Internet'''
'''SGW''' '''MME''' '''PGW'''
<100 control plane <-->
<5 user plane <-->
30-100 backbone <-->
variable routing <-->
</code>

==== Control Plane Latency ====
<code>
'''GPRS EDGE HSPA HSPA+ LTE LTE-Advanced'''
'''Latency''' [ms] <1000 <1000 <500 <200 <100 <50
</code>

=== Image Compression ===

==== Image Formats ====

{| class="wikitable"
|-
! Format !! Lossy !! Transparency support !! Supported by Firefox
|-
| JPEG || Yes || No || Yes
|-
| JPEG XR || Yes || Yes || No
|-
| JPEG 2000 || Yes || Yes || No
|-
| PNG || No || Yes || Yes
|-
| JNG || Yes || Yes || No
|-
| WebP || Yes || Yes || No
|-
| BMP || No || Yes || Yes
|-
|}

==== Sample ====

The sample used for the tests below was made using images from 20 different webpages from the Alexa's top 200 most visited websites list.

{| class="wikitable"
|-
! Format !! Number of images !! Total size (MB)
|-
| JPEG || 325 || 12
|-
| PNG || 817 || 14
|-
| GIF || 325 || 1.8
|}

==== PNG Compression ====

The PNG sample size is 12383KB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| optipng -o 1 || 11560 || 7 || 70
|-
| optipng -o 5 || 10940 || 12 || 481
|-
| [http://pngquant.org/ pngquant] || 5169 || 59 || 69
|-
| pngcrush || 10997 || 12 || 90
|-
| pngcrush + optipng -o 5 || 10280 || 17 || 541
|-
| WebP (quality = 90) || 4741 || 62 || 47
|-
| WebP (quality = 100) || 8321 || 33 || 48
|-
|}

==== JPEG Compression ====

The sample size is 10846kB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| mozjpeg || 9663 || 11 || 53
|-
| mozjpeg (quality = 90) || 8571 || 21 || -
|-
| jpegoptim || 10497 || 4 || 39
|-
| WebP (quality = 100) || 17551 || -61 || 42
|-
| WebP (quality = 90) || 8316 || 24 || 40
|-
|}

==== GIF Compression ====

The sample size is 846kB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| gifsicle || 805 || 5 || 16
|-
|}

==== Animated GIF Compression ====

It is possible to convert animated GIFs to WebM videos (as [http://gfycat.com/ gfycat.com] does), this can be done with a straightforward ffmpeg command. The processing time can be high but the size reduction is huge!

Some stats from a small set of 10 animated GIFs:
{| class="wikitable"
|-
! Original sample size (kB) !! New sample size (kB) !! Average compression (%) !! Total time (s) !! Median time (s)
|-
| 25162 || 1177 || 95 || 25 || 2.46
|-
|}

{| class="wikitable"
|-
! Image size !! Time (s)
|-
| 531K || 0.301
|-
| 844K || 0.719
|-
| 992K || 0.847
|-
| 1.1M || 2.726
|-
| 1.3M || 2.199
|-
| 1.5M || 2.831
|-
| 1.7M || 1.497
|-
| 3.5M || 5.223
|-
| 4.5M || 2.777
|-
| 8.0M || 6.604
|-
|}

=== Common Misconceptions ===
==== Losing radio link means losing TCP connection ====
The PGW terminates the TCP/UDP connections, application level connectivity is not tied to the physical radio link.

==== A device session is tied to an IP ====
The PGW provides NAT services, a device can be mapped to multiple IP/port combinations or multiple devices can share the same IP.

==== Bandwidth is the decisive factor for page load speed ====
Bandwidth gives the upper bound on transmission speed under optimal conditions. In realistic environments, considering the speed of light and the shortcomings of the transmission protocol, ''latency'' is the limiting factor for the maximum throughput.

== Tracking Bugs ==
The Node.js based proxy implementation is maintained on GitHub: [https://github.com/mozilla/node-gonzales node-gonzales]. Please use it to report issues and make pull requests.

Required '''features''' that Janus depends on are tracked here
* {{bug|378637}} - Add support for connecting to HTTP proxy over HTTPS
* {{bug|366559}} - Firefox/Gecko should support LZMA as an HTTP transfer-encoding method
* {{bug|1010068}} - Disable OCSP on Firefox for Android

Known '''issues''' affecting Janus are tracked here
* {{bug|1014589}} - Fennec crashes on page load when connected with SPDY proxy

== System Overview ==
Janus is a secure proxy server that accepts requests via the SPDY protocol and serves compressed content directly from its cache, if available.

<code>
Browser <-SPDY-> Janus <-HTTP/HTTPS/SPDY-> Web
| |
Proxy <-> Cache <-?-> Compressor <-> Loader
</code>

Routing requests through a SPDY proxy does require less open TCP connections (multiplexing), decreases packet sizes (header compression) and reduces the number of packets (header caching) compared to HTTP/1. Low bandwidth and high latency connections should benefit from this.

The current trend on the Web shows an increase in average page size, especially due to high-resolution images. Additional image compression and downsizing should decrease the bandwidth requirements and enable faster page loads. To counteract the introduced processing overhead and boost load times for slow pages, we add intermediate caching for the compressed results.

== Experimental Setups ==
To evaluate the system, we test each component in isolation first.
We identify third-party systems suitable for integration into the prototype and evaluate them. All qualified systems will be considered for integration or as the base of our final implementation.

=== SPDY Proxy ===
Currently, only Google Chrome supports secure SPDY proxies, but it requires valid certificates, which makes it unsuitable for testing. We bypass the missing support by using '''shrpx''' in client mode as a ''forward proxy'', and that way simulate SPDY support on any client.

Dependencies:
* Firefox/Fennec
* Spdylay (http://tatsuhiro-t.github.io/spdylay)
* Squid (http://www.squid-cache.org)
* [alternative] Node SPDY proxy (https://github.com/igrigorik/node-spdyproxy)

Here are some experimental setups.

==== P1 ====
In this setup, we connect a desktop browser via HTTP with a local forwarding proxy, which establishes a SPDY connection with the remote secure SPDY proxy. The secure SDPY proxy connects to a local web proxy to fetch the documents.

<code>
Desktop Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| | | | |
Firefox <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
</code>

==== P2 ====
This setup is similar to P1, but running on a mobile client. This requires either direct secure SPDY proxy support on the mobile browser, or the usage of a local forwarding proxy running on the mobile device.

<code>
Mobile Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| | | | |
Fennec <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
</code>

=== Compression ===
''TODO''

=== Caching ===
''TODO''

== Performance Analysis ==
Let's collect some ideas here how to profile our prototypes.

Could be useful:
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests Talos page load tests]

== References ==
* [http://www.ietf.org/rfc/rfc2186.txt Internet Cache Protocol (ICP)]
* [http://www.ietf.org/rfc/rfc2187.txt Application of Internet Cache Protocol (ICP)]
* [http://tools.ietf.org/html/rfc2616 HTTP/1.1]
* [http://tools.ietf.org/html/draft-ietf-httpbis-http2-13 HTTP/2]
* [http://tools.ietf.org/id/draft-agl-tls-nextprotoneg-00.txt Transport Layer Security (TLS) Next Protocol Negotiation Extension (NPN)]
* [http://www.chromium.org/spdy/spdy-proxy Chromium SPDY Proxy]
* [http://tatsuhiro-t.github.io/spdylay/ Spdylay - SPDY C Library]
* [https://github.com/igrigorik/node-spdyproxy Node.js SPDY Forwarding Proxy]
* [http://nginx.org/ NGINX - Reverse Proxy]
* [http://developer.att.com/application-resource-optimizer AT&T ARO]
* [http://www.igvita.com/posa/high-performance-networking-in-google-chrome/ High Performance Networking in Google Chrome]
* [http://commondatastorage.googleapis.com/io-2013/presentations/263%20-%20io-radio-performance.pdf Mobile Performance from Radio Up] ([https://developers.google.com/events/io/sessions/327864330 video])
* [http://www.igvita.com/2012/06/25/spdy-and-secure-proxy-support-in-google-chrome/ Secure SPDY Proxy Support in Google Chrome]
* [http://amazonsilk.wordpress.com/2011/09/28/introducing-amazon-silk/ Amazon Silk Browser]
* [http://chimera.labs.oreilly.com/books/1230000000545 High Performance Browser Networking]
* [https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-ssl/ Experience report on CDNs and SPDY]
* [https://developers.google.com/speed/docs/best-practices/caching Caching best practices]

Mobile/Janus

2014-07-07T16:57:14Z

Tglek:

Janus is a project to find and evaluate ways to enable more efficient mobile browsing.

== Goals ==
The goal of Janus is to verify whether we can enhance the mobile browsing experience by achieving the following improvements in a practical way:
* Reduce radio time
* Reduce page load times
* Reduce bandwidth requirements
* '''Increase user privacy'''
* Increase responsiveness for slow sites

Ways we hope to accomplish some of these things:
* Recompress images to be smaller without noticeable visual impact
* Compress text (js, html) streams (gzip, bzip2, lzma?)
* Caching
* Reduce HTTPS round trips
* Move safe browsing support to the proxy
* Link prefetching

Other crazy stuff we could do:
* Man-in-the-middle HTTPS traffic only for images (so they get recompressed) with user consent
* Convert animated GIF to H264/WebM/whatever. gfycat.com does this now with apparently good results
* Pre-Shumwayize Flash content
* Automatic readability mode for some sites
* Ad blocking
** Only obnoxious ads?
** Replace with our own?
** Only block ones that track?
* Add support for adaptive streaming (MPEG-DASH) for servers that don't support it. Transcode to lower bitrates.
** Proxy HLS sites as MPEG-DASH
* Opera Mini-like pre-rendering of pages

Taras crazy stuff:
* http://zsync.moria.org.uk/ content that changes a little bit, but frequently
<code>
Currently we send if-modified-since:
We could add header supports-zsync:

This would make server reply with a .zsync which the client would then use to submit range: requests.
</code>
* asset bundling/inlining for cold loads
** server can determine when client got nothing in cache(eg no if-modified-since), do spdy-push from a preset profile
** server could spdy-push partial files(eg only css that gets used) + zsync of rest of contents so client can request missing pieces for other pages on the website(this can hugely reduce transfer sizes)...If i remember correctly from https://bugzilla.mozilla.org/show_bug.cgi?id=834865 70%+ of css is leftover cruft...Note this might be easier to do having sites be aware of this and bundling a library that supports having css/js broken up into multiple files & supports waiting on missing pieces

== Background ==

=== Transport Layer Security (TLS) ===
==== TLS handshake ====
<code>
'''Time''' [ms] '''Client''' '''Server'''
30 SYN -->
60 <-- SYN ACK
90 ACK -->
120 <-- Certificate
150 ChangeChiperSpec -->
180 <-- ChangeCipherSpec
210 Data -->
240 <-- Data
</code>

=== 3G/4G ===
==== LTE Mobile Network ====
<code>
'''Latency''' [ms] '''Type''' '''Device''' '''RAN''' '''Core Network''' '''Internet'''
'''SGW''' '''MME''' '''PGW'''
<100 control plane <-->
<5 user plane <-->
30-100 backbone <-->
variable routing <-->
</code>

==== Control Plane Latency ====
<code>
'''GPRS EDGE HSPA HSPA+ LTE LTE-Advanced'''
'''Latency''' [ms] <1000 <1000 <500 <200 <100 <50
</code>

=== Image Compression ===

==== Image Formats ====

{| class="wikitable"
|-
! Format !! Lossy !! Transparency support !! Supported by Firefox
|-
| JPEG || Yes || No || Yes
|-
| JPEG XR || Yes || Yes || No
|-
| JPEG 2000 || Yes || Yes || No
|-
| PNG || No || Yes || Yes
|-
| JNG || Yes || Yes || No
|-
| WebP || Yes || Yes || No
|-
| BMP || No || Yes || Yes
|-
|}

==== Sample ====

The sample used for the tests below was made using images from 20 different webpages from the Alexa's top 200 most visited websites list.

{| class="wikitable"
|-
! Format !! Number of images !! Total size (MB)
|-
| JPEG || 325 || 12
|-
| PNG || 817 || 14
|-
| GIF || 325 || 1.8
|}

==== PNG Compression ====

The PNG sample size is 12383KB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| optipng -o 1 || 11560 || 7 || 70
|-
| optipng -o 5 || 10940 || 12 || 481
|-
| [http://pngquant.org/ pngquant] || 5169 || 59 || 69
|-
| pngcrush || 10997 || 12 || 90
|-
| pngcrush + optipng -o 5 || 10280 || 17 || 541
|-
| WebP (quality = 90) || 4741 || 62 || 47
|-
| WebP (quality = 100) || 8321 || 33 || 48
|-
|}

==== JPEG Compression ====

The sample size is 10846kB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| mozjpeg || 9663 || 11 || 53
|-
| mozjpeg (quality = 90) || 8571 || 21 || -
|-
| jpegoptim || 10497 || 4 || 39
|-
| WebP (quality = 100) || 17551 || -61 || 42
|-
| WebP (quality = 90) || 8316 || 24 || 40
|-
|}

==== GIF Compression ====

The sample size is 846kB.

{| class="wikitable sortable"
|-
! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
|-
| gifsicle || 805 || 5 || 16
|-
|}

==== Animated GIF Compression ====

It is possible to convert animated GIFs to WebM videos (as [http://gfycat.com/ gfycat.com] does), this can be done with a straightforward ffmpeg command. The processing time can be high but the size reduction is huge!

Some stats from a small set of 10 animated GIFs:
{| class="wikitable"
|-
! Original sample size (kB) !! New sample size (kB) !! Average compression (%) !! Total time (s) !! Median time (s)
|-
| 25162 || 1177 || 95 || 25 || 2.46
|-
|}

{| class="wikitable"
|-
! Image size !! Time (s)
|-
| 531K || 0.301
|-
| 844K || 0.719
|-
| 992K || 0.847
|-
| 1.1M || 2.726
|-
| 1.3M || 2.199
|-
| 1.5M || 2.831
|-
| 1.7M || 1.497
|-
| 3.5M || 5.223
|-
| 4.5M || 2.777
|-
| 8.0M || 6.604
|-
|}

=== Common Misconceptions ===
==== Losing radio link means losing TCP connection ====
The PGW terminates the TCP/UDP connections, application level connectivity is not tied to the physical radio link.

==== A device session is tied to an IP ====
The PGW provides NAT services, a device can be mapped to multiple IP/port combinations or multiple devices can share the same IP.

==== Bandwidth is the decisive factor for page load speed ====
Bandwidth gives the upper bound on transmission speed under optimal conditions. In realistic environments, considering the speed of light and the shortcomings of the transmission protocol, ''latency'' is the limiting factor for the maximum throughput.

== Tracking Bugs ==
The Node.js based proxy implementation is maintained on GitHub: [https://github.com/mozilla/node-gonzales node-gonzales]. Please use it to report issues and make pull requests.

Required '''features''' that Janus depends on are tracked here
* {{bug|378637}} - Add support for connecting to HTTP proxy over HTTPS
* {{bug|366559}} - Firefox/Gecko should support LZMA as an HTTP transfer-encoding method
* {{bug|1010068}} - Disable OCSP on Firefox for Android

Known '''issues''' affecting Janus are tracked here
* {{bug|1014589}} - Fennec crashes on page load when connected with SPDY proxy

== System Overview ==
Janus is a secure proxy server that accepts requests via the SPDY protocol and serves compressed content directly from its cache, if available.

<code>
Browser <-SPDY-> Janus <-HTTP/HTTPS/SPDY-> Web
| |
Proxy <-> Cache <-?-> Compressor <-> Loader
</code>

Routing requests through a SPDY proxy does require less open TCP connections (multiplexing), decreases packet sizes (header compression) and reduces the number of packets (header caching) compared to HTTP/1. Low bandwidth and high latency connections should benefit from this.

The current trend on the Web shows an increase in average page size, especially due to high-resolution images. Additional image compression and downsizing should decrease the bandwidth requirements and enable faster page loads. To counteract the introduced processing overhead and boost load times for slow pages, we add intermediate caching for the compressed results.

== Experimental Setups ==
To evaluate the system, we test each component in isolation first.
We identify third-party systems suitable for integration into the prototype and evaluate them. All qualified systems will be considered for integration or as the base of our final implementation.

=== SPDY Proxy ===
Currently, only Google Chrome supports secure SPDY proxies, but it requires valid certificates, which makes it unsuitable for testing. We bypass the missing support by using '''shrpx''' in client mode as a ''forward proxy'', and that way simulate SPDY support on any client.

Dependencies:
* Firefox/Fennec
* Spdylay (http://tatsuhiro-t.github.io/spdylay)
* Squid (http://www.squid-cache.org)
* [alternative] Node SPDY proxy (https://github.com/igrigorik/node-spdyproxy)

Here are some experimental setups.

==== P1 ====
In this setup, we connect a desktop browser via HTTP with a local forwarding proxy, which establishes a SPDY connection with the remote secure SPDY proxy. The secure SDPY proxy connects to a local web proxy to fetch the documents.

<code>
Desktop Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| | | | |
Firefox <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
</code>

==== P2 ====
This setup is similar to P1, but running on a mobile client. This requires either direct secure SPDY proxy support on the mobile browser, or the usage of a local forwarding proxy running on the mobile device.

<code>
Mobile Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| | | | |
Fennec <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
</code>

=== Compression ===
''TODO''

=== Caching ===
''TODO''

== Performance Analysis ==
Let's collect some ideas here how to profile our prototypes.

Could be useful:
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests Talos page load tests]

== References ==
* [http://www.ietf.org/rfc/rfc2186.txt Internet Cache Protocol (ICP)]
* [http://www.ietf.org/rfc/rfc2187.txt Application of Internet Cache Protocol (ICP)]
* [http://tools.ietf.org/html/rfc2616 HTTP/1.1]
* [http://tools.ietf.org/html/draft-ietf-httpbis-http2-13 HTTP/2]
* [http://tools.ietf.org/id/draft-agl-tls-nextprotoneg-00.txt Transport Layer Security (TLS) Next Protocol Negotiation Extension (NPN)]
* [http://www.chromium.org/spdy/spdy-proxy Chromium SPDY Proxy]
* [http://tatsuhiro-t.github.io/spdylay/ Spdylay - SPDY C Library]
* [https://github.com/igrigorik/node-spdyproxy Node.js SPDY Forwarding Proxy]
* [http://nginx.org/ NGINX - Reverse Proxy]
* [http://developer.att.com/application-resource-optimizer AT&T ARO]
* [http://www.igvita.com/posa/high-performance-networking-in-google-chrome/ High Performance Networking in Google Chrome]
* [http://commondatastorage.googleapis.com/io-2013/presentations/263%20-%20io-radio-performance.pdf Mobile Performance from Radio Up] ([https://developers.google.com/events/io/sessions/327864330 video])
* [http://www.igvita.com/2012/06/25/spdy-and-secure-proxy-support-in-google-chrome/ Secure SPDY Proxy Support in Google Chrome]
* [http://amazonsilk.wordpress.com/2011/09/28/introducing-amazon-silk/ Amazon Silk Browser]
* [http://chimera.labs.oreilly.com/books/1230000000545 High Performance Browser Networking]
* [https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-ssl/ Experience report on CDNs and SPDY]
* [https://developers.google.com/speed/docs/best-practices/caching Caching best practices]

Performance/Avoid SQLite In Your Next Firefox Feature

2014-03-20T22:18:43Z

Tglek: /* I/O Patterns */

== Introduction ==
Many Firefox developers see SQLite as a default choice for storing any non-trivial amount of data, this wiki explains why that view is incorrect.

SQLite may seem like a very appealing option when first designing a feature. It offers the familiar and powerful SQL language, our codebase has nice C++ and JavaScript APIs for it, and it's already used in many places in the codebase. However, when choosing storage formats, we have to keep in mind that SQLite is a powerful and reliable database and that it comes with a lot of '''hidden complexity'''. It's very tempting to take the abstractions it offers at face value and to ignore the complexity happening under the hood. As a result, we have repeatedly seen SQLite become a source of performance problems. This isn't an indictment of SQLite itself -- any other relational embedded DB would pose the same challenges.

SQLite DBs are simply too complex to be used for relatively simple data storage needs. The list below outlines some of the ways SQLite can backfire on developers.

== SQLite Pitfalls ==

=== Storage Footprint ===

* Features that regularly add data to the DB (e.g. visited URIs), but have no expiration policy for the data, can easily cause the DB to balloon into tens or hundreds of MBs. This is particularly undesirable on mobile devices
* WAL journals have been known to grow very large, e.g. {{bug|609122}}, {{bug|608422}}
* Every index contains a complete copy of the indexed data. Creating indexes on lengthy text fields will cause very large indexes to be stored on disk
* Using the default microsecond precision for timestamps causes unnecessary bloat

=== I/O Patterns ===

* Your database file can suffer from logical/internal fragmentation due to common SQL patterns
** Eg appending data to columns with indexes causes interleaved index and table data. This means both table and index scans are non-sequential until a VACUUM is performed.
** Appending data to multiple tables interleaves tables too.
** Deleting data does not shrink the database, just marks pages as unused.
* Physical/external fragmentation can also happen due to SQLite using inappropriate defaults
** SQLite does not preallocate files by default. Appending data to an SQLite in multiple sessions often means the OS has to start a new data block that's not adjacent to the previous one.
** By default sqlite will grow/shrink the database file by |page_size|. This behavior causes new data blocks to be allocated too. This is especially problematic on OSX, Linux XFS.
** SQLite [https://www.sqlite.org/c3ref/c_fcntl_chunk_size.html#sqlitefcntlchunksize CHUNK_SIZE] feature is a way to minimize this problem
Factory-default SQLite [http://www.sqlite.org/pragma.html#pragma_page_size page size] is 1024 bytes. When the Mozilla default page size was changed to 32KB in {{bug|416330}}, there was a [http://taras.glek.net/blog/2013/06/28/new-performance-people/ 4x reduction in SQLite IO waits] according to Telemetry. This is likely due to a reduction in syscalls and OS readahead.
* Performance can [http://sqlite.1065341.n5.nabble.com/Coping-with-database-growth-fragmentation-td44781.html#a44782 significantly degenerate over time]; scheduling periodic [https://sqlite.org/lang_vacuum.html rebuilds of the DB] is necessary.
** Note that this "vacuuming" only deals with internal fragmentation, not the external [filesystem] fragmentation
* SQLite uses fsync's to guarantee transaction durability and enable recovery from crashes. fsyncs can be very expensive, and happen relatively frequently in the default rollback-journal mode. This performance/reliability trade-off might not be necessary for simple storage needs.
* JSON files or log files will show better I/O patterns almost every time, especially if they're compressed and read/written in entirety each time

=== Memory Usage ===

* By default, our SQLite wrapper uses a [http://hg.mozilla.org/mozilla-central/file/444714c3820a/storage/src/mozStorageConnection.cpp#l48 max of 2MB memory cache per connection].
** This cache size may be too large for Fennec and B2G, especially if there are multiple connections. The cache size should be adjusted with [[#Important Pragmas|PRAGMAs]]

=== CPU Usage ===

* Users have found certain Firefox features using 15% of their CPU time because SQL statements were constantly being executed in the background
* Lock contention can occur when using the DB from multiple threads
** e.g. VACCUM-ing the DB on one thread while executing SQL queries on another

=== Battery Life ===

* Same as CPU, important on mobile

=== Unintended Main-Thread I/O ===

Main-thread SQL is a known evil, and luckily there are only a few major sources of it left in the codebase, but the [http://telemetry.mozilla.org/slowsql/ Slow SQL dashboard] shows that developers are still getting bitten by main thread I/O from unexpected sources:

* PRAGMA statements to set the cache size are done on the main thread, but if this is the first transaction of the session, it can trigger SQLite crash recovery operations. This is particularly bad if using the default WAL journal size. Also note that "crashes" are particularly common on mobile where we never have clean exits.
* Some addons access Firefox DBs directly using main thread SQL

=== Contending for Access to Storage with the Main Thread ===

Bad SQLite performance off the main thread also degrades Firefox responsiveness because it can contend with the main thread for access to storage. Sadly our code still does main-thread I/O, and additionally, there will always be main-thread IO from add-ons and swapping.

* Even seemingly simple SQL queries can take multiple seconds to execute
** e.g. in {{bug|966469}} a user reports the following statement taking 22 seconds to execute: <code>DELETE FROM moz_pages WHERE id NOT IN (SELECT DISTINCT(pid) FROM moz_subresources);</code>

=== UX Impact of Slow Queries ===

Slow DB operations hurt UX. For example, you might have noticed that the AwesomeBar sometimes takes a very long time to fetch any auto-complete suggestions.

* The schemas have to be designed carefully and the DB has to be maintained periodically
* Fragmentation causes extra seeks, and the problem is made worse by magnetic hard disks and cheap or old SSDs. Vacuuming DBs helps, but vacuums are resource-intensive and scheduling them can be tricky. See the [[#More Tips|Tips]] section

== How to Store Your Data ==

* If you need to store a small amount of data (less than 1MB), you should use JSON files, and do all your I/O off the main thread. Simple solutions with linear worst-case performance are ideal.
** If you're working with larger amounts of data (roughly on the order of 1MB), you should compress the JSON data with Snappy before writing it to disk
** You can use OS.File's [https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread#OS.File.writeAtomic%28%29 writeAtomic()] method. More information on writeAtomic [http://dutherenverseauborddelatable.wordpress.com/2014/02/05/is-my-data-on-the-disk-safety-properties-of-os-file-writeatomic/ in this blog post]
** If your workload involves a lot of strings, don't use SQLite. Store your data in external files.
* For larger datasets or when SQL is absolutely necessary, use SQLite.
** Make sure you understand [https://sqlite.org/docs.html how SQLite works], carefully design your schemas and then profile your implementation.
** IndexedDB is implemented on top of SQLite and has additional issues

NOTE: ''We're currently working on an intermediate solution based on log-storage which will reduce the amount of data written for small modifications to data already on disk.''

== How to Best Use SQLite If You Really Need To ==

=== Important Pragmas ===
See [http://www.sqlite.org/pragma.html the list of SQLite pragmas], the defaults are probably not OK. We already over-ride some defaults in the [http://mxr.mozilla.org/mozilla-central/source/db/sqlite3/src/moz.build moz.build file] and [http://mxr.mozilla.org/mozilla-central/source/storage/src/mozStorageConnection.cpp#622 mozStorageConnection.cpp].

* Use a [https://www.sqlite.org/wal.html write-ahead log] for fewer fsyncs
** <code>PRAGMA journal_mode = WAL;</code>
* Set the DB cache size based on the device specs, and consider adjusting it dynamically based on memory pressure on mobile devices
** <code>PRAGMA cache_size = X;</code>
* For large DBs prone to fragmentation, pre-allocate a large empty DB initially by increasing the DB's growth increment
** This can be done by calling Connection::SetGrowthIncrement() which changes the value of SQLite's <code>SQLITE_FCNTL_CHUNK_SIZE</code>. See {{bug|581606}} for reference.

=== More Tips ===

* It should go without saying that you should '''never execute SQL on the main thread'''
** Use the [https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Sqlite.jsm SQLite.jsm wrapper] for DB operations from JavaScript, it's fully asynchronous and off-main-thread
** For native code, use the [https://mxr.mozilla.org/mozilla-central/source/storage/public/mozIStorageAsyncConnection.idl async methods]
* Model how large your DB will grow on desktop/mobile. Report the DB size & memory use to [https://developer.mozilla.org/en-US/docs/Performance/Adding_a_new_Telemetry_probe Telemetry] to confirm your estimates were accurate. '''Implement an expiration policy!'''
* Profile your implementation and measure the number of SQL operations performed over the course of a typical session if it’s an “always-on” feature and you suspect it might cause performance issues
* Don’t index on text fields, index on hashes of text fields. Indexes can become very large.
* Don't use the default precision for timestamps (microseconds). Desktop apps are unlikely to ever require this level of precision
* Maintain your DB with [https://sqlite.org/lang_vacuum.html VACUUMs] during Firefox version upgrades or during idle times. Run [http://www.sqlite.org/lang_analyze.html ANALYZE] periodically if necessary
** You should have a vacuum plan from the get-go. '''This is a requirement for all new code'''.

Performance/Avoid SQLite In Your Next Firefox Feature

2014-03-20T22:12:20Z

Tglek: /* I/O Patterns */

== Introduction ==
Many Firefox developers see SQLite as a default choice for storing any non-trivial amount of data, this wiki explains why that view is incorrect.

SQLite may seem like a very appealing option when first designing a feature. It offers the familiar and powerful SQL language, our codebase has nice C++ and JavaScript APIs for it, and it's already used in many places in the codebase. However, when choosing storage formats, we have to keep in mind that SQLite is a powerful and reliable database and that it comes with a lot of '''hidden complexity'''. It's very tempting to take the abstractions it offers at face value and to ignore the complexity happening under the hood. As a result, we have repeatedly seen SQLite become a source of performance problems. This isn't an indictment of SQLite itself -- any other relational embedded DB would pose the same challenges.

SQLite DBs are simply too complex to be used for relatively simple data storage needs. The list below outlines some of the ways SQLite can backfire on developers.

== SQLite Pitfalls ==

=== Storage Footprint ===

* Features that regularly add data to the DB (e.g. visited URIs), but have no expiration policy for the data, can easily cause the DB to balloon into tens or hundreds of MBs. This is particularly undesirable on mobile devices
* WAL journals have been known to grow very large, e.g. {{bug|609122}}, {{bug|608422}}
* Every index contains a complete copy of the indexed data. Creating indexes on lengthy text fields will cause very large indexes to be stored on disk
* Using the default microsecond precision for timestamps causes unnecessary bloat

=== I/O Patterns ===

* Your database file can suffer from logical/internal fragmentation due to common SQL patterns
** Eg appending data to columns with indexes causes interleaved index and table data. This means both table and index scans are non-sequential until a VACUUM is performed.
** Appending data to multiple tables interleaves tables too.
** Deleting data does not shrink the database, just marks pages as unused.
* Physical/external fragmentation can also happen due to SQLite using inappropriate defaults
** e.g. the factory-default SQLite [http://www.sqlite.org/pragma.html#pragma_page_size page size] is 1024 bytes. When the default page size was changed to 32KB in {{bug|416330}}, there was a [http://taras.glek.net/blog/2013/06/28/new-performance-people/ 4x reduction in SQLite IO waits] according to Telemetry
* Performance can [http://sqlite.1065341.n5.nabble.com/Coping-with-database-growth-fragmentation-td44781.html#a44782 significantly degenerate over time]; scheduling periodic [https://sqlite.org/lang_vacuum.html rebuilds of the DB] is necessary.
** Note that this "vacuuming" only deals with internal fragmentation, the filesystem has its own fragmentation
* SQLite uses fsync's to guarantee transaction durability and enable recovery from crashes. fsyncs can be very expensive, and happen relatively frequently in the default rollback-journal mode. This performance/reliability trade-off might not be necessary for simple storage needs.
* JSON files or log files will show better I/O patterns almost every time, especially if they're compressed and read/written in entirety each time

=== Memory Usage ===

* By default, our SQLite wrapper uses a [http://hg.mozilla.org/mozilla-central/file/444714c3820a/storage/src/mozStorageConnection.cpp#l48 max of 2MB memory cache per connection].
** This cache size may be too large for Fennec and B2G, especially if there are multiple connections. The cache size should be adjusted with [[#Important Pragmas|PRAGMAs]]

=== CPU Usage ===

* Users have found certain Firefox features using 15% of their CPU time because SQL statements were constantly being executed in the background
* Lock contention can occur when using the DB from multiple threads
** e.g. VACCUM-ing the DB on one thread while executing SQL queries on another

=== Battery Life ===

* Same as CPU, important on mobile

=== Unintended Main-Thread I/O ===

Main-thread SQL is a known evil, and luckily there are only a few major sources of it left in the codebase, but the [http://telemetry.mozilla.org/slowsql/ Slow SQL dashboard] shows that developers are still getting bitten by main thread I/O from unexpected sources:

* PRAGMA statements to set the cache size are done on the main thread, but if this is the first transaction of the session, it can trigger SQLite crash recovery operations. This is particularly bad if using the default WAL journal size. Also note that "crashes" are particularly common on mobile where we never have clean exits.
* Some addons access Firefox DBs directly using main thread SQL

=== Contending for Access to Storage with the Main Thread ===

Bad SQLite performance off the main thread also degrades Firefox responsiveness because it can contend with the main thread for access to storage. Sadly our code still does main-thread I/O, and additionally, there will always be main-thread IO from add-ons and swapping.

* Even seemingly simple SQL queries can take multiple seconds to execute
** e.g. in {{bug|966469}} a user reports the following statement taking 22 seconds to execute: <code>DELETE FROM moz_pages WHERE id NOT IN (SELECT DISTINCT(pid) FROM moz_subresources);</code>

=== UX Impact of Slow Queries ===

Slow DB operations hurt UX. For example, you might have noticed that the AwesomeBar sometimes takes a very long time to fetch any auto-complete suggestions.

* The schemas have to be designed carefully and the DB has to be maintained periodically
* Fragmentation causes extra seeks, and the problem is made worse by magnetic hard disks and cheap or old SSDs. Vacuuming DBs helps, but vacuums are resource-intensive and scheduling them can be tricky. See the [[#More Tips|Tips]] section

== How to Store Your Data ==

* If you need to store a small amount of data (less than 1MB), you should use JSON files, and do all your I/O off the main thread. Simple solutions with linear worst-case performance are ideal.
** If you're working with larger amounts of data (roughly on the order of 1MB), you should compress the JSON data with Snappy before writing it to disk
** You can use OS.File's [https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread#OS.File.writeAtomic%28%29 writeAtomic()] method. More information on writeAtomic [http://dutherenverseauborddelatable.wordpress.com/2014/02/05/is-my-data-on-the-disk-safety-properties-of-os-file-writeatomic/ in this blog post]
** If your workload involves a lot of strings, don't use SQLite. Store your data in external files.
* For larger datasets or when SQL is absolutely necessary, use SQLite.
** Make sure you understand [https://sqlite.org/docs.html how SQLite works], carefully design your schemas and then profile your implementation.
** IndexedDB is implemented on top of SQLite and has additional issues

NOTE: ''We're currently working on an intermediate solution based on log-storage which will reduce the amount of data written for small modifications to data already on disk.''

== How to Best Use SQLite If You Really Need To ==

=== Important Pragmas ===
See [http://www.sqlite.org/pragma.html the list of SQLite pragmas], the defaults are probably not OK. We already over-ride some defaults in the [http://mxr.mozilla.org/mozilla-central/source/db/sqlite3/src/moz.build moz.build file] and [http://mxr.mozilla.org/mozilla-central/source/storage/src/mozStorageConnection.cpp#622 mozStorageConnection.cpp].

* Use a [https://www.sqlite.org/wal.html write-ahead log] for fewer fsyncs
** <code>PRAGMA journal_mode = WAL;</code>
* Set the DB cache size based on the device specs, and consider adjusting it dynamically based on memory pressure on mobile devices
** <code>PRAGMA cache_size = X;</code>
* For large DBs prone to fragmentation, pre-allocate a large empty DB initially by increasing the DB's growth increment
** This can be done by calling Connection::SetGrowthIncrement() which changes the value of SQLite's <code>SQLITE_FCNTL_CHUNK_SIZE</code>. See {{bug|581606}} for reference.

=== More Tips ===

* It should go without saying that you should '''never execute SQL on the main thread'''
** Use the [https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Sqlite.jsm SQLite.jsm wrapper] for DB operations from JavaScript, it's fully asynchronous and off-main-thread
** For native code, use the [https://mxr.mozilla.org/mozilla-central/source/storage/public/mozIStorageAsyncConnection.idl async methods]
* Model how large your DB will grow on desktop/mobile. Report the DB size & memory use to [https://developer.mozilla.org/en-US/docs/Performance/Adding_a_new_Telemetry_probe Telemetry] to confirm your estimates were accurate. '''Implement an expiration policy!'''
* Profile your implementation and measure the number of SQL operations performed over the course of a typical session if it’s an “always-on” feature and you suspect it might cause performance issues
* Don’t index on text fields, index on hashes of text fields. Indexes can become very large.
* Don't use the default precision for timestamps (microseconds). Desktop apps are unlikely to ever require this level of precision
* Maintain your DB with [https://sqlite.org/lang_vacuum.html VACUUMs] during Firefox version upgrades or during idle times. Run [http://www.sqlite.org/lang_analyze.html ANALYZE] periodically if necessary
** You should have a vacuum plan from the get-go. '''This is a requirement for all new code'''.

Telemetry/Reboot

2013-10-23T23:19:15Z

Tglek: /* Telemetry Backend */

Goal: Fast, Robust server & frontend operational by '''Sep 30th''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.

==Telemetry Backend==
Migration
* Nov 1: Start porting dashboard generation script to hadoop
* Nov 7: Start running ^ script
* Nov 30: Turn off hadoop

==Telemetry Dashboard==
Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones:
*Sep 1: Add median, percentiles, 90% ready
*Sep 15: ^ finished
*Sep 15: Produce a second set of dashboards(same codebase, probably different http server) that are being fed data from AWS

Open bugs: https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Telemetry%20Dashboard&list_id=7702794

==Misc==
Notes:
* Original etherpad https://etherpad.mozilla.org/telemetry-reboot

Performance/Frontend

2013-09-09T21:31:02Z

Tglek:

Telemetry/Reboot

2013-08-20T21:48:29Z

Tglek: /* Telemetry Dashboard */

Goal: Fast, Robust server & frontend operational by '''Sep 31st''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.

==Telemetry Backend==
Server(https://github.com/mreid-moz/telemetry-server/) requirements:
* Ability to process 10x incoming packet rates of metrics telemetry infrastructure on a single AWS instance: 2400req/s with 30K HTTP POST packets. Fall
* Server should be bandwidth-limited, not CPU.
* Server should make data available for map/reduce immediately. Fallback goal: 5min latency. In Q4 we'd like to use something like heka to make dashboards use live data(0min lag).
* Graphite reporting: valid packet rates for each channel, stats on packet sizes, etc

Server Milestones:
* '''Sep 2''': Ability temporarily(1 hour?) point telemetry dns at AWS, forwarding telemetry to metrics cluster. Ideally we'd be able to do this at bouncer level so no changes are needed to accomplish forwarding
* '''Sep 16''': ^ should be feeding both servers (metrics and AWS) until cutover date.

==Telemetry Dashboard==
Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones:
*Sep 1: Add median, percentiles, 90% ready
*Sep 15: ^ finished
*Sep 15: Produce a second set of dashboards(same codebase, probably different http server) that are being fed data from AWS

Open bugs: https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Telemetry%20Dashboard&list_id=7702794

==Misc==
Notes:
* Original etherpad https://etherpad.mozilla.org/telemetry-reboot

Telemetry/Reboot

2013-08-20T21:43:21Z

Tglek:

Goal: Fast, Robust server & frontend operational by '''Sep 31st''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.

==Telemetry Backend==
Server(https://github.com/mreid-moz/telemetry-server/) requirements:
* Ability to process 10x incoming packet rates of metrics telemetry infrastructure on a single AWS instance: 2400req/s with 30K HTTP POST packets. Fall
* Server should be bandwidth-limited, not CPU.
* Server should make data available for map/reduce immediately. Fallback goal: 5min latency. In Q4 we'd like to use something like heka to make dashboards use live data(0min lag).
* Graphite reporting: valid packet rates for each channel, stats on packet sizes, etc

Server Milestones:
* '''Sep 2''': Ability temporarily(1 hour?) point telemetry dns at AWS, forwarding telemetry to metrics cluster. Ideally we'd be able to do this at bouncer level so no changes are needed to accomplish forwarding
* '''Sep 16''': ^ should be feeding both servers (metrics and AWS) until cutover date.

==Telemetry Dashboard==
Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones:
*Sep 1: Add median, percentiles, 90% ready
*Sep 15: ^ finished

Open bugs: https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Telemetry%20Dashboard&list_id=7702794

==Misc==
Notes:
* Original etherpad https://etherpad.mozilla.org/telemetry-reboot

Telemetry/Reboot

2013-08-20T21:39:10Z

Tglek:

Goal: Fast, Robust server & frontend operational by '''Sep 31st''' able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating. Metrics should be able to retire telemetry portion of mango cluster in October.

==Telemetry Backend==
Server(https://github.com/mreid-moz/telemetry-server/) requirements:
* Ability to process 10x incoming packet rates of metrics telemetry infrastructure on a single AWS instance: 2400req/s with 30K HTTP POST packets. Fall
* Server should be bandwidth-limited, not CPU.
* Server should make data available for map/reduce immediately. Fallback goal: 5min latency. In Q4 we'd like to use something like heka to make dashboards use live data(0min lag).

Server Milestones:
* '''Sep 2''': Ability temporarily(1 hour?) point telemetry dns at AWS, forwarding telemetry to metrics cluster. Ideally we'd be able to do this at bouncer level so no changes are needed to accomplish forwarding
* '''Sep 16''': ^ should be feeding both servers (metrics and AWS) until cutover date.

==Telemetry Dashboard==
Dashboard(https://github.com/mozilla/telemetry-dashboard/) Milestones

Notes
* Original etherpad https://etherpad.mozilla.org/telemetry-reboot

Telemetry/Reboot

2013-08-20T21:34:17Z

Tglek:

Telemetry/Reboot

2013-08-20T20:43:26Z

Tglek:

Telemetry/Reboot

2013-08-20T20:40:35Z

Tglek:

Telemetry/Reboot

2013-08-20T20:30:43Z

Tglek: Created page with "Goal: Fast, Robust server & frontend operational by Sep 31st able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updatin..."

Goal: Fast, Robust server & frontend operational by Sep 31st able to accept and graph telemetry data. This means that as of Oct 1 http://metrics.mozilla.com/ will stop updating.
* Server
**
* Original etherpad https://etherpad.mozilla.org/telemetry-reboot

Performance/Frontend

2013-08-06T12:49:12Z

Tglek:

Gavin:
* {{bug|699854}} - Refactor Downloads APIs to use async Storage [Q3]
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs [some in Q3]
* {{bug|899401}} - Reduce thumbnailing overhead [Q3?]
Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms - Q3

Taras:
* {{bug|853388}} + {{bug|853389}} - JSON+Async addon manager - Early Q3
* {{bug|838577}} - Async session-store - Q3(looks like Q4 due to new 'e10s' design)

Performance/Frontend

2013-07-31T00:44:27Z

Tglek:

Platform/2013-Q3-Goals

2013-07-24T20:08:42Z

Tglek: /* Perf */

=== Platform ===
==== [[Platform/2013-Goals|2013 General Goals]] ====
# Games/Media/Social/Location
# Performance
# Development velocity/productivity
# Firefox OS
# Firefox Desktop
# Firefox Android
# Overall Software Quality

=== GFX ===

==== Goals ====

* GPU/shader pipeline
** {{ok|{{nbug|687187}}, {{nbug|827229}} - SkiaGL canvas support on Android. [2,6]}} (Trains 24, 25)
** {{ok|{{nbug|883445}} - B2G canvas rendering using SkiaGL}}
** {{ok|Implement CSS Filters}}
* OMT* Goals
** {{ok|Complete Off-Main Thread Animation feature & tests (Train 25)}}
** {{ok|{{nbug|883441}}, {{nbug|883433}} - Async pan-zoom, including sub-frames. [2,3,5]}} (Trains 25, 26)}}
* Moz2D (Azure)
** {{ok|Land Direct2D 1.1 backend (Train 25)}}
** {{ok|{{nbug|891660}} Land OS X CoreGraphics backend (Train 25)}}
** {{ok|Land Cairo backend (Train 26)}}
* {{ok|The GFX team will resolve all actionable security bugs within six weeks of their entering our components and receiving sec-high or sec-critical designation. [7]}} (All trains)

==== Wishlist ====
* {{wishlist|WebGL performance improvements as driven by the games team. [2,6]}} (Trains 24,25)
* {{wishlist|More robust downloadable blocklisting solution}}
* {{wishlist|Fix WebGL security bugs.}} (Train 25)

==== Investigations ====
* {{wishlist|{{nbug|882383}} - Improve layer heuristics}}
* {{wishlist|{{nbug|883448}} - Scrolled content uploading (e.g., tiling vs. buffer rotation) on mobile}}
* {{wishlist|{{nbug|886315}} - Make GL shader pipeline more flexible}}
* {{wishlist|Realistic scrolling performance tests}}
* {{wishlist|Tiling everywhere?}}

==== Moz2D & OMTC Path Reference ====

{|
! align="center" valign="top" style="width: 50%;" | OMT*
! align="center" valign="top" style="width: 50%;" | Moz2D (Azure)
|-
| valign="top" |
* OMT Image Animation
* OMTC Mac
* OMTC Metro
----
* OMTC Windows
* OMTC SW
* OMT Texture Uploads
* OMT Painting
* Remove non-OMT code
| valign="top" |
* Moz2D Skia
* Moz2D SkiaGL
* Moz2D CG
* Moz2D Cairo
|}

=== Layout ===
* {{ok|vertical text goal TO WRITE}}
* {{ok|text performance goal TO WRITE}}
* {{ok|CSS variables ({{bug|773296}}) landed on mozilla-central}}
* {{ok|CSS flexbox pagination ({{bug|811024}}) and multiline flexbox ({{bug|702508}}) landed on mozilla-central}}
* {{ok|working incremental layout updates in Servo, for the features Servo implements}}
* {{ok|have our own CSS parsing and selector matching code in Servo}}

=== Media ===
* {{ok|Get YouTube Media Source Extensions demo player working.}}
* {{ok|Use an integrated MP4 demuxer instead of depending on the platform. It is needed to reduce crashing on Windows 7, improve security on Vista and for adding H.264 support on Mac.}}
* {{ok|Ship Web Audio in Firefox 24.}}

=== DOM ===
* {{ok|Incremental cycle collection landed and on by default (mccr8)}}
* {{done|Convert Navigator to new bindings (bzbarsky)}}
* {{ok|Convert Location to new bindings (peterv)}}
* {{ok|Make Window use WebIDL quickstubs (peterv)}}
* {{ok|Codegen cleanup for workers (khuey)}}
* {{ok|Make exposing QI in chrome on WebIDL things op-in (peterv)}}
* {{ok|Update document.register to latest spec revision ({{bug|856140}}) (blake / wchen)}}
* {{ok|Implement ShadowRoot event retargeting algorithm ({{bug|887541}}) (blake / wchen)}}
* {{ok|ShadowRoot <shadow> element ({{bug|887538}}) (blake / wchen)}}
* {{ok|Implement HTML imports ({{bug|877072}}) (blake / wchen)}}
* {{ok|Identify approach to port exsiting APIs to port to workers. Port 1-3 APIs (khuey)}}
* {{ok|Improve editor performance by eliminating use of tons of Range objects (smaug)}}
* {{ok|Codegenerator for WebIDL-only DOM events(smaug)}}
* {{ok|Allow child processes to share a plugin-container per plugin (johns)}}
* {{ok|Implement and land the One Logger to Rule Them All ({{bug|onelogger}}) (jlebar)}}
* {{ok|Clean up DMD reports for B2G under normal use(jlebar)}}
* {{ok| Convert SimplePush to webidl (nikhil)}}
* {{ok| Drive SimplePush web standard (nikhil)}}
* {{ok| Help with porting APIs to workers (nikhil)}}
* {{ok| Try to implement background services with system messages integration, including out of process for B2G}} (nikhil)
* {{ok| Finalize the patch for per-document,docshell,window event queue (olli)}}

=== WebAPI ===
* {{done|land Promises on m-c ({{bug|856410}}) (baku)}}
* {{ok|land DataStore API on m-c ({{bug|871445}}) (baku)}}
* {{ok|Investigate event pages (marcosc)}}
* {{ok|publish [http://www.w3.org/2012/sysapps/manifest/ manifest format] (marcosc)}}
* {{ok|publish policy on exposing APIs to the web (overholt)}}
* {{ok|have draft text on Streams (annevk)}} (someone from Google is handling IO streams already)
* {{ok|make URL parsing (in URL Standard) idempotent (annevk)}}
* {{ok|update URL test suite (annevk)}}
* {{ok|expose Notification objects associated with an origin (annevk)}}
* {{ok|Review and land sync IDB in workers}} (janv)
* {{ok|Multiprocess FileHandle and FileSystem API}} (janv & co.)
* {{ok|Ship storage.js}} (mounir)
* {{ok|Implement NavigationController}} (ehsan & annevk)
* {{ok|Improve IndexedDB error messages}} (bent)
* {{ok|Convert IndexedDB to WebIDL bindings}} (ehsan, baku, janv)
* {{ok|Implement several new IndexedDB features that came out of Google meetup}} (bent)
* {{ok|Get ScreenOrientation API to last call}} (mounir)
* {{wishlist|Integrate localStorage with temporary storage}} (janv)
* {{wishlist|Fully support OS.File worker needs}} (bent)
* {{wishlist|Make IPDL work for XPCOM event loops}} (bent)

=== JS ===

=== Accessibility ===
* FFOS: Finish all the pieces for developer-ready a11y.(eeejay)
** {{ok|Minimal Gaia accessibility lockscreen/homescreen/settings.}}
** {{ok|Screen Reader pref in developer settings.}}
** {{ok|Pico library in gonk / Pico engine in gecko layer.}}
* {{ok|FF OS: Identify and file bugs for core GAIA apps needing accessibility work (MarcoZ)}}
* {{ok|Stability: to include crashes and new asserts.}} Tracking {{Bug|888531}}.(tbsaunde)
* {{ok|Resolve all actionable sec-moderate or higher a11y bugs within six weeks of their sec-rating.}}(tbsaunde)
* {{wishlist|Fix key/priority community bugs including text work. Tracking}} {{Bug|887794}}.(surkov)

=== Perf ===
* 50% startup speedup(per eideticker nytimes test) for "GET /" on Galaxy Nexus (4s atm) - bug 807322
* Async collection + writing of session-store - bug 83857
* Fsync-less prefs - bug 866238

=== Networking ===

* {{ok|Disk Cache - new cache API and file system layer passing tests on gum branch (includes crash recovery and being jank-free: does not include in-memory index or appcache integration) (honza, michal)}}
* {{ok|HTTP/2 Node.js Test Server (Nick, Gabor (GSoC Student))}}
* {{ok|B2G Per Application traffic meter support (jduell)}}
* {{ok|off-main-thread ODA clients raster image loading ({{bug|867755}}) and Gzipped content ({{bug|882996}})(sworkman)}}
* {{ok|Disable Speculative Connections for RFC 1918 connections ({{bug|853423}}) (Steve)}}
* {{ok|Resource Timing (http://www.w3.org/TR/resource-timing/) (Steve/Intern)}}
* {{ok|TLS false start (mcmanus)}}
* {{ok|HTTP/2 draft-04 (or subsequent as apropos) to gather interop experience and work with IETF and refinements. (mcmanus/hurley)}}
* {{ok|Deploy empirical test of DNS viability as HTTP/2 hint in testing channels. (mcmanus)}}
* {{ok|Predictive Support - Startup, Pageload, Link hover, rel prefetch (hurley)}}

=== Mobile ===

Code Review

2013-07-11T17:07:56Z

Tglek: /* Potential Bugzilla Improvements */

== Reviewer Guidelines ==

# '''Prioritize reviews highly.''' Reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't write code any more due to prioritizing reviews over coding, appoint more reviewers.
# '''Communicate.''' If you are an active contributor, you should respond to a review request within one working day of getting it, either with a review, a deadline by which you promise to do the review, or a polite refusal. If you think a patch is lower priority than your other work communicate that.
# '''Silence is not an option.''' If you think saying nothing is better than admitting than you won't get to the patch for a while, that's being [https://en.wikipedia.org/wiki/Passive-aggressive_behavior passive-aggressive]. This is not a good way to build a happy coding community. Holding back bad code is a feature, not a bug - so do it, quickly and politely.

== Patch Author Guidelines ==

There are some things that make it easier to get a good, prompt review, and to improve the quality of the patch you submit. Help your reviewers review your patches quickly by making patches easy to review:

* '''Split out mass-changes or mechanical changes''' into a separate patch from the substantive changes.
* '''Separate patches into conceptually-separate pieces''' for review purposes (even if you then later collapse them into a single changeset to push), if technically possible. Any time you're requesting review from multiple people on a single huge diff, chance are splitting it might have been a good idea.
* '''Make the summary of the bug reflect the problem''' rather than some particular symptom of it, so there's a clear description of what the patch is trying to fix
* '''Write good commit messages''' that describe what's changing between old and new code (which, if it can't be summarized in less than about 100-150 characters, should have a short summary on the first line and a longer description on later lines)
* '''Write good code comments''' that describe the state of the new code, and if the patch is of nontrivial size, point to the important comments in the non-first lines of the commit message
* '''Address all comments from previous reviews''' (either by fixing as requested, or explaining why you haven't) before requesting another review. It's very common to just ignore (miss?) some of the review comments, which means the reviewer then needs to triple-check that all the things they pointed out got fixed.
* '''Provide an interdiff''' for second and subsequent patches when requesting review for them. This is so that the reviewer can verify that the changes you made match what they asked for. (Bugzilla's interdiff is totally unsuitable for this purpose, unfortunately, because it fails so often because it doesn't have access to the original code.)
* '''If your request only relates to part of the patch, say so clearly.'''
* '''Don't request review from people who are labelled as being away''' in Bugzilla.

== Potential Bugzilla Improvements ==
* In addition to r+/r- add a '''Review ETA''' field where the requested reviewer can specify when the patch will get reviewed. Bugzilla should send reminder emails based on that flag.
** Reminders should be set to every 24-hours by default
** it should be possible to get reminders for multiple people (eg for managers to track review latency of whole team)
** reminders should specify how long has it been since reviewer last commented in the bug
* Provide information about reviewers when asking for r?
** Add a notion of being away to Bugzilla accounts, or integrate with a Time Off tracking app.
** Display a user's number of outstanding reviews
* Add a 'low-priority' patch flag
* Some integration with a formatting checker

Code Review

2013-07-11T17:07:03Z

Tglek: /* Potential Bugzilla Improvements */

== Reviewer Guidelines ==

# '''Prioritize reviews highly.''' Reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't write code any more due to prioritizing reviews over coding, appoint more reviewers.
# '''Communicate.''' If you are an active contributor, you should respond to a review request within one working day of getting it, either with a review, a deadline by which you promise to do the review, or a polite refusal. If you think a patch is lower priority than your other work communicate that.
# '''Silence is not an option.''' If you think saying nothing is better than admitting than you won't get to the patch for a while, that's being [https://en.wikipedia.org/wiki/Passive-aggressive_behavior passive-aggressive]. This is not a good way to build a happy coding community. Holding back bad code is a feature, not a bug - so do it, quickly and politely.

== Patch Author Guidelines ==

There are some things that make it easier to get a good, prompt review, and to improve the quality of the patch you submit. Help your reviewers review your patches quickly by making patches easy to review:

* '''Split out mass-changes or mechanical changes''' into a separate patch from the substantive changes.
* '''Separate patches into conceptually-separate pieces''' for review purposes (even if you then later collapse them into a single changeset to push), if technically possible. Any time you're requesting review from multiple people on a single huge diff, chance are splitting it might have been a good idea.
* '''Make the summary of the bug reflect the problem''' rather than some particular symptom of it, so there's a clear description of what the patch is trying to fix
* '''Write good commit messages''' that describe what's changing between old and new code (which, if it can't be summarized in less than about 100-150 characters, should have a short summary on the first line and a longer description on later lines)
* '''Write good code comments''' that describe the state of the new code, and if the patch is of nontrivial size, point to the important comments in the non-first lines of the commit message
* '''Address all comments from previous reviews''' (either by fixing as requested, or explaining why you haven't) before requesting another review. It's very common to just ignore (miss?) some of the review comments, which means the reviewer then needs to triple-check that all the things they pointed out got fixed.
* '''Provide an interdiff''' for second and subsequent patches when requesting review for them. This is so that the reviewer can verify that the changes you made match what they asked for. (Bugzilla's interdiff is totally unsuitable for this purpose, unfortunately, because it fails so often because it doesn't have access to the original code.)
* '''If your request only relates to part of the patch, say so clearly.'''
* '''Don't request review from people who are labelled as being away''' in Bugzilla.

== Potential Bugzilla Improvements ==
* In addition to r+/r- add a '''Review ETA''' field where the requested reviewer can specify when the patch will get reviewed. Bugzilla should send reminder emails based on that flag.
** Reminders should be 24hours by default
** it should be possible to get reminders for multiple people (eg for managers to track review latency of whole team)
** reminders should specify how long has it been since reviewer last commented in the bug
* Provide information about reviewers when asking for r?
** Add a notion of being away to Bugzilla accounts, or integrate with a Time Off tracking app.
** Display a user's number of outstanding reviews
* Add a 'low-priority' patch flag
* Some integration with a formatting checker

Code Review

2013-07-10T22:12:08Z

Tglek: /* Bugzilla Improvements */

== Reviewer Guidelines ==

# Prioritize. Reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't write code anymore due to prioritizing reviews over coding, grow more reviewers.
# Communicate. If you are an active contributor, you should not leave r? patches sitting in your queue without feedback. "I will review this next week because I'm (busy reviewing ___ this week|away at conference). If you think a patch is lower priority than your other work communicate that.
# Silence is not an option. If you think saying nothing is better than admitting than you wont get to the patch for a while(Holding back bad code is a feature, not a bug, do it politely), that's passive aggressiveness (https://en.wikipedia.org/wiki/Passive-aggressive_behavior). This is not a good way to build a happy coding community.

== Patch-author Guidelines ==

There are some things that make it easier to get a good, prompt review, and to improve the quality of the patch you submit. Help your reviewers review your patches quickly by making patches easy to review:

* Split mass-changes or mechanical changes into a separate patch from the substantive changes.
* If possible, separate patches into conceptually-separate pieces for review purposes (even if you then later collapse them into a single changeset to push). Any time you're requesting review from multiple people on a single huge diff, chance are splitting it might have been a good idea.
* Actually address the review comments before requesting another review. It's very common to just ignore (miss?) some of the review comments, which means the reviewer then needs to triple-check that all the things they pointed out got fixed.
* When requesting a second review on a patch, provide an interdiff so that the reviewer can just verify that the changes you made match what they asked for. Bugzilla's interdiff is totally unsuitable for this purpose, unfortunately, because it fails so often.
* When requesting review or feedback on just part of the patch, make that very clear.
* don't request review from people who are labeled as being away in Bugzilla and expect fast turnaround

== Bugzilla Improvements ==
* In addition to r+/r- add eta flag to specify when the patch will get reviewed. Bugzilla should send reminder emails based on that flag.
* Provide information about reviewers when asking for r?
** Add a notion of being away to bz-accounts.
** Display number of outstanding reviews
* Add a 'low-priority' patch flag
* Some integration with a formatting checker

Code Review

2013-07-10T22:11:26Z

Tglek: /* Bugzilla Improvements */

Code Review

2013-07-10T21:57:07Z

Tglek: /* Patch-author Guidelines */

Code Review

2013-07-10T21:55:05Z

Tglek: /* Bugzilla Improvements */

Code Review

2013-07-10T21:52:51Z

Tglek: /* Patch-author Guidelines */

Code Review

2013-07-10T21:50:49Z

Tglek: /* Patch-author Guidelines */

Code Review

2013-07-10T21:49:41Z

Tglek: /* Reviewer Guidelines */

Code Review

2013-07-10T21:48:17Z

Tglek:

== Reviewer Guidelines ==

# Prioritize. Realize that reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't write code anymore due to prioritizing reviews over coding, grow more reviewers.
# Communicate. If you are an active contributor, you should not leave r? patches sitting in your queue without feedback. "I will review this next week because I'm (busy reviewing ___ this week|away at conference). If you think a patch is lower priority than your other work communicate that.
# Silence is not an option. If you think saying nothing is better than admitting than you wont get to the patch for a while(Holding back bad code is a feature, not a bug, do it politely), that's passive aggressiveness (https://en.wikipedia.org/wiki/Passive-aggressive_behavior). This is not a good way to build a happy coding community.

== Patch-author Guidelines ==

There are some things that make it easier to get a good, prompt review, and to improve the quality of the patch you submit. Help your reviewers help you, and don't be surprised if you get asked about one of the following:

* small patches, functionally divided;

== Bugzilla Improvements ==
* In addition to r+/r- add eta flag to specify when the patch will get reviewed. Bugzilla should send reminder emails based on that flag.

Code Review

2013-07-10T21:45:23Z

Tglek: Created page with "== Reviewer Guidelines == # Prioritize. Realize that reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't..."

== Reviewer Guidelines ==

# Prioritize. Realize that reviewing code is more valuable than writing code as it results in higher overall project activity. If you find you can't write code anymore due to prioritizing reviews over coding, grow more reviewers.
# Communicate. If you are an active contributor, you should not leave r? patches sitting in your queue without feedback. "I will review this next week because I'm (busy reviewing ___ this week|away at conference). I think bugzilla could use some improvements there. If you think a patch is lower priority than your other work communicate that.
# Silence is not an option. If you think saying nothing is better than admitting than you wont get to the patch for a while(Holding back bad code is a feature, not a bug, do it politely), that's passive aggressiveness (https://en.wikipedia.org/wiki/Passive-aggressive_behavior). This is not a good way to build a happy coding community.

== Patch-author Guidelines ==

There are some things that make it easier to get a good, prompt review, and to improve the quality of the patch you submit. Help your reviewers help you, and don't be surprised if you get asked about one of the following:

* small patches, functionally divided;

PlatformProfiler/Roadmap

2013-07-10T19:01:49Z

Tglek: /* Profilable B2G Nightlies */

= Projects =
== Profilable Fennec Nightlies ==
The goal of this project is being able to profile nightly Fennec build without compiling a local build.

Owner: :sewardj

Bugs
* Breakpad speed improvements
* Breakpad unwind accuracy

== Profilable B2G Nightlies ==
The goal of this project is being able to profile nightly B2G build without compiling a local build.

Owner: :jld,:sewardj

Bugs
* Custom ARM Unwinder
* Use Breakpad on B2G
* Optimize profiler memory usage

== Profiling Plugins ==
The goal of this project is being able to profile plugins.

Owner: :gfritzsche, :benwa, :vlandan

Bugs
* Bug 853864 - Expose IPC message information to profiler
* Bug 881899 - Deploy a private SymbolicationServer for obfuscated Flash symbols
* Front end improvements to display IPC message information
== Monitoring I/O ==
The goal of this project is report I/O operation with a proper callstack.

Owner: :aklotz

Bugs
* Bug 867757 - Provide a way to insert arbitrary data in a profile
* Capture the callstacks for markers
* Symbolicate marker callstacks
* Front end improvements to markers and callstacks.
== Profiling Painting/Compositing (also Gaming) ==
The goal of this project is being able to report (1) What is triggering repaints, (2) Frame boundaries, (3) Frame budgets and frames taking above 15ms, (4) Samples related to frame for both the content and compositor thread.

Owner: :BenWa

Bugs
* Bug 867757 - Provide a way to insert arbitrary data in a profile
* Instrument the frame transactions
* Capture callstack of code requesting repaints
* Add a new widget to show frame budgets
* Complex time range selection in the front end
== Cleopatra ==
The goal of this project is to improve cleopatra.

Owner: :BenWa, :mstange

Bugs
* (Various support feature for projects listed here)
* Better marker UI
* Fix local storage
* Code refactoring
* Tests
== Multi-thread Profiling ==
The goal of this project is to display more useful profiles for tasks that coordinate work between threads

Owner: :BenWa

Bugs
* Improvements to thread selection and filtering
* Bug 867757 - Provide a way to insert arbitrary data in a profile
* Show locking?
== Power Profiling ==
Bugs
* Bug 769431 - Correlate performance data with Ivy Bridge MSR_PPX_ENERGY_STATUS for power usage profiles
== Experimental/Low Priority ==
=== Video Profiling ===
The goal of this project is display a video capture along side of the project requested. Similar to eideticker profiles.

Owner: :BenWa

Bugs
* Either improve automation with screencasting software or build in system to copy composited frames.

=== Resuming unwinds from PseudoFrames ===
The goal of this project is to experiment with from a failed unwind from a pseudo frames and/or using previous pseudo frames to copy from a successfull unwind to speed up unwinding.

Performance/Frontend

2013-06-27T18:21:57Z

Tglek:

Gavin:
* {{bug|853549}} - Remove main-thread I/O from password manager
* {{bug|699854}} - Refactor Downloads APIs to use async Storage
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs
* {{bug|876002}} - [meta] Remove nsIFormHistory2 so no synchronous form history code remains
* {{bug|843853}} - Newtab page slows down tab-open animation

Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms

Taras:
* {{bug|853388}} + {{bug|853389}} - JSON+Async addon manager - Early Q3
* {{bug|838577}} - Async session-store - Q3

Performance/Frontend

2013-06-27T18:18:14Z

Tglek:

Performance/Frontend

2013-06-14T21:27:13Z

Tglek:

Gavin:
* {{bug|853549}} - Remove main-thread I/O from password manager
* {{bug|699854}} - Refactor Downloads APIs to use async Storage
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs
* {{bug|876002}} - [meta] Remove nsIFormHistory2 so no synchronous form history code remains
* {{bug|843853}} - Newtab page slows down tab-open animation

Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms

Taras:
* {{bug|853388}} + {{bug|853389}} - JSON+Async addon manager
* {{bug|702559}} - Create a pure-async mozIStorageAsyncConnection
* {{bug|838577}} - Async session-store

Performance/Frontend

2013-06-14T21:10:42Z

Tglek:

Gavin:
* {{bug|853549}} - Remove main-thread I/O from password manager
* {{bug|699854}} - Refactor Downloads APIs to use async Storage
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs
* {{bug|876002}} - [meta] Remove nsIFormHistory2 so no synchronous form history code remains
* {{bug|843853}} - Newtab page slows down tab-open animation

Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms

Taras:
* {{bug|853388}} + {{bug|853389}} - JSON+Async addon manager
* {{bug|702559}} - Create a pure-async mozIStorageAsyncConnection

Performance/Frontend

2013-06-07T23:00:10Z

Tglek:

Gavin:
* {{bug|853549}} - Remove main-thread I/O from password manager
* {{bug|699854}} - Refactor Downloads APIs to use async Storage
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs
* {{bug|876002)) - [meta] Remove nsIFormHistory2 so no synchronous form history code remains
* {{bug|843853}} - Newtab page slows down tab-open animation

Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms

Taras:
* {{bug|853388}} + {{bug|853389}} - JSON+Async addon manager
* {{bug|702559}} - Create a pure-async mozIStorageAsyncConnection
* {{bug|872421}} [Chrome Workers] Provide a module loader for chrome workers

Performance/Frontend

2013-06-07T22:56:56Z

Tglek:

Gavin:
* {{bug|853549}} - Remove main-thread I/O from password manager
* {{bug|699854}} - Refactor Downloads APIs to use async Storage
* {{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs
* {{bug|876002)) - [meta] Remove nsIFormHistory2 so no synchronous form history code remains
* {{bug|843853}} - Newtab page slows down tab-open animation

Dolske:
* {{bug|355063}} - Password manager does not work on script-generated forms

Taras:
* {{bug|853388}} - {{bug|853389}} - JSON+Async addon manager
* {{bug|702559}} - Create a pure-async mozIStorageAsyncConnection
* {{bug|872421}} [Chrome Workers] Provide a module loader for chrome workers

Performance/Frontend

2013-06-07T22:46:37Z

Tglek:

Performance/Frontend

2013-06-07T22:27:31Z

Tglek:

Performance/Frontend

2013-06-07T22:27:13Z

Tglek: Created page with "Gavin: {{bug|853549}} - Remove main-thread I/O from password manager {{bug|699854}} - Refactor Downloads APIs to use async Storage {{bug|699850}} - Refactor all synchronous hi..."

Gavin:
{{bug|853549}} - Remove main-thread I/O from password manager
{{bug|699854}} - Refactor Downloads APIs to use async Storage
{{bug|699850}} - Refactor all synchronous history APIs to use async Storage APIs

Javascript:SpiderMonkey:2013Projects

2013-05-14T23:51:30Z

Tglek: /* Projects Under Consideration */

== Projects Under Consideration ==

* Generational Garbage Collection, Terrence Cole/Steve Fink/Jon Coppeard, https://wiki.mozilla.org/Javascript:SpiderMonkey:ProjectGenerationGarbageCollection
* OdinMonkey (Asm.js), Luke/Sean/Hannes, https://wiki.mozilla.org/Javascript:SpiderMonkey:OdinMonkey
* JavaScript Parrellelism, Sean and Luke
** Parrellel JS, Mozilla Research
** Parrellel Arrays
** ArrayBuffer sharing
** Thread support
* EcmaScript 6 Features, Jason Orendorff, https://wiki.mozilla.org/ES6_plans
** Modules - Edy & Jason
* EcmaScript 6 Testing, Jeff Walden, https://wiki.mozilla.org/User:Waldo/ES6_testing
* Baseline, Kannan/Jan
** Optimize generators and proxies in baseline
** Make Ion use Baseline information
* BinaryData: {{bug|578700}}
* Internationalization, Jeff Walden
** Write up Norbert's work and API - https://wiki.mozilla.org/User:Waldo/Internationalization_API
* Memory Usage
** Lazy Byte Code Analysis, Brian Hackett, {{bug|678037}}
** Script Analysis, Kannan Vijayan, {{bug|864502}}
* ARM, Marty Rosenberg
** Assembly Buffer changes - needed for asm.js
** Add support for all 32 double registers
** thumb2 backend - http://en.wikipedia.org/wiki/ARM_architecture#Thumb-2
*** new assembler for thumb2
** Add a peephole pass: {{bug|734830}}
** Support for operand2 and fused-multiply-add: {{bug|777659}}
** Support for pre and post increment
** Front-end on top of the macro assembler to get yarr to use it (will likely generate better code)
* Type Inference, Brian Hackett
** Reduce TI memory usage {{bug|804676}}
* FirefoxOS
** Partner benchmark optimizations
** B2G Optimizations, Nicolas
* DevTools, Jim Blandy
** Debugability of full stack
** JSD2 Trace Profiler {{bug|797876}}
** Profiler
*** Crash Bugs - Typically seen while toggling the profiler and/or private browsing
**** {{bug|802798}}
**** {{bug|836938}}
**** {{bug|832812}} fixed
**** {{bug|788495}}
*** prototype/static function name {{bug|838265}}
*** multithreaded {{bug|863703}}
*** Column info {{bug|785922}}
* Performance
** Faster DOM <==> JIT Interaction
** Shumway and IonMonkey JIT code-gen caching, Till Schneidereit
*** Memory cloning
*** Disk Caching
* Hygiene
** Remove Jaegermonkey code from the tree

Performance/FHR

2013-02-14T19:39:46Z

Tglek:

1. ''' JSON vs SQLite storage ''' The SQLite schema used in FHR is overly complex and seems over-engineered and has a degree of redundancy.
We could simplify FHR storage by storing data in JSON files, this will cut down amount of IO and fsyncs.
We should flush out current session data on shutdown(without fsync to delay shutdown).

2. ''' Flushing prefs ''' FHR currently uses browser prefs to store data such as lastPingTime, lastSubmitID, last submission success, etc. It will need to explicitly flush the prefs store to disk for this information to survive a crash. Currently, prefs can only be flushed on the main thread, and this is likely to be a source of jank as typical pref files are sizeable.

3. ''' User-activity tracking ''' The FHR user-activity handler should be implemented natively. The overhead of one JS event running every 5 seconds (while the user is interacting with the browser) is not likely to cause any user-visible jank, but this functionality really belongs in platform code. Once this data provider is moved to native code, FHR will have 1) "pull type" data providers that are only polled at shutdown or upload time, and 2) "push type" providers that record specific user actions (e.g. Google searches). If we could move the "push type" providers out of FHR, FHR would only have to be in memory at shutdown or submission time.

4. ''' FHR event polling ''' Currently, FHR wakes up every minute to check if it needs to submit a report, expire any data, etc. This can be made neater by simply scheduling wake-ups for a specific time in the future.

5. ''' Chrome workers ''' Any disk or network operations should be moved entirely to a chrome worker to avoid unintentionally slowing down the main thread. For example, serializing 6-months worth of FHR data on the main thread before passing the string to Necko's thread-pool could cause jank (we've seen this problem with session restore). FHR can then use XHR from a worker to do uploads instead of re-implementing the functionality on the main thread & using XPCOM to open network channels.

6. ''' Spinning of event loop at shutdown ''' We shouldn't be spinning the event loop at shutdown. It seems event-loop spinning was added to allow an FHR upload to finish before Firefox shuts down, but we could achieve the same effect by cancelling the upload and updating state atomically at the end of the upload.

7. ''' Telemetry probes ''' FHR should track its first-run initialization time in Telemetry separately from subsequent initialization times. We should also change the existing Telemetry probes to accurately track the amount of time spent on FHR operations instead of tracking the wall clock time elapsed from issuing an asynchronus FHR operation to the operation's completion handler running. It's causing the probes to over-estimate the amount of time required by FHR operations.

8. ''' Other comments ''' The current uses of Task.jsm are hard to read, they should be rewritten with better naming conventions or task closures.

Performance/Snappy/Dashboard

2013-01-28T22:53:54Z

Tglek: /* Fixed in the last week. */

=Snappy Dashboard=

==Resolved in the last 3 weeks.==
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "21d",
"x_axis_field": "component"
}
</bugzilla>
==Fixed in last 3 weeks==
<bugzilla>
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "21d"
}
</bugzilla>

==Changed in the last week.==
<bugzilla type="count" display="bar">
{
"quicksearch": "status:new,assigned,reopened,unconfirmed",
"whiteboard": "[snappy",
"changed_after": "7d",
"x_axis_field": "component"
}
</bugzilla>
<bugzilla>
{
"quicksearch": "whiteboard:[snappy status:new,assigned,reopened,unconfirmed",
"changed_after": "7d"
}
</bugzilla>

Performance/Snappy/Dashboard

2013-01-28T22:53:31Z

Tglek: /* Changed in the last week. */

=Snappy Dashboard=

==Resolved in the last 3 weeks.==
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "21d",
"x_axis_field": "component"
}
</bugzilla>
==Fixed in the last week.==
<bugzilla>
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "21d"
}
</bugzilla>

==Changed in the last week.==
<bugzilla type="count" display="bar">
{
"quicksearch": "status:new,assigned,reopened,unconfirmed",
"whiteboard": "[snappy",
"changed_after": "7d",
"x_axis_field": "component"
}
</bugzilla>
<bugzilla>
{
"quicksearch": "whiteboard:[snappy status:new,assigned,reopened,unconfirmed",
"changed_after": "7d"
}
</bugzilla>

Performance/Snappy/Dashboard

2013-01-28T22:52:45Z

Tglek: /* Resolved in the last week. */

=Snappy Dashboard=

==Resolved in the last 3 weeks.==
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "21d",
"x_axis_field": "component"
}
</bugzilla>
==Changed in the last week.==
<bugzilla>
{
"whiteboard": "[snappy",
"resolution": "fixed",
"chfield": "resolution",
"changed_after": "7d"
}
</bugzilla>

==Changed in the last week.==
<bugzilla type="count" display="bar">
{
"quicksearch": "status:new,assigned,reopened,unconfirmed",
"whiteboard": "[snappy",
"changed_after": "7d",
"x_axis_field": "component"
}
</bugzilla>
<bugzilla>
{
"quicksearch": "whiteboard:[snappy status:new,assigned,reopened,unconfirmed",
"changed_after": "7d"
}
</bugzilla>

Platform/2012-11-27

2012-11-27T19:00:29Z

Tglek: /* Snappy [Weekly] */

[[Platform/{{#time: Y-m-d | {{SUBPAGENAME}} -1 week}}|« previous week]] | [[Platform|index]] | [[Platform/{{#time: Y-m-d | {{SUBPAGENAME}} +1 week}}|next week »]]

<div class="h-event vevent">
'''Platform Meeting Details'''
* Tuesday {{#time: Y-m-d | {{SUBPAGENAME}} }} - 11:00 am <abbr class="value" title="-0700">Pacific</abbr>
{{conf|95312}}
* [https://v.mozilla.com/flex.html?roomdirect.html&key=UK1zyrd7Vhym Warp Core Vidyo Room] / SFO-Boardroom
* join irc.mozilla.org [irc://irc.mozilla.org/planning #planning] for back channel
</div>

== Moztrap ==

==Notices/Schedule==
{| class="wikitable"
|-
| <center><big>[https://bugzilla.mozilla.org/buglist.cgi?type0-7-0=notequals;value0-7-0=%2B;field0-3-0=cf_status_firefox19;type0-1-0=notequals;type0-5-0=notequals;value0-5-0=disabled;value0-4-0=verified;field0-1-0=cf_status_firefox19;field0-0-0=cf_tracking_firefox19;type0-4-0=notequals;columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate;field0-6-0=cf_status_firefox19;value0-3-0=unaffected;field0-7-0=cf_tracking_firefox18;query_format=advanced;value0-2-0=fixed;value0-6-0=verified%20disabled;value0-1-0=wontfix;type0-3-0=notequals;field0-2-0=cf_status_firefox19;field0-5-0=cf_status_firefox19;field0-4-0=cf_status_firefox19;type0-6-0=notequals;type0-0-0=equals;value0-0-0=%2B;type0-2-0=notequals;list_id=5004437 27 bugs]</big> (17)</center>
| <center><big>[https://bugzilla.mozilla.org/buglist.cgi?type0-1-0=notequals;type0-5-0=notequals;value0-5-0=disabled;value0-4-0=verified;list_id=4754086;field0-1-0=cf_status_firefox18;field0-0-0=cf_tracking_firefox18;value0-3-0=unaffected;value0-6-0=verified%20disabled;value0-1-0=wontfix;field0-5-0=cf_status_firefox18;type0-0-0=equals;value0-0-0=%2B;type0-2-0=notequals;field0-3-0=cf_status_firefox18;type0-4-0=notequals;columnlist=bug_severity%2Cpriority%2Cop_sys%2Cassigned_to%2Cbug_status%2Cresolution%2Cshort_desc%2Cchangeddate;field0-6-0=cf_status_firefox18;query_format=advanced;value0-2-0=fixed;type0-3-0=notequals;field0-2-0=cf_status_firefox18;field0-4-0=cf_status_firefox18;type0-6-0=notequals 85 bugs]</big> (67)</center>
|-
! scope="row" | Unresolved Aurora Trackers (non-security, not tracked for Beta)
! scope="row" | Unresolved Beta Trackers (non-security)
|}

* Firefox 18.0 beta 1 went out the door yesterday
* Firefox 19.0a2 updates were enabled yesterday
* Necessity for a 17.0.1 discussed in the channel meeting today
* Firefox 18.0 beta 2 is going to build today
** Team nags sent - please prioritize tracked FF18 bugs asap. Think of the holidays.

* 17.0.1 potential drivers:
** {{bug|813763}} – Menu items like Addons Manager not working in FF 17 due to Tab Mix Plus
*** We've marked TMP as incompatible with FF17
*** TMP removed from 'featured' addons for now, and also SUMO points to dev version which has fix in it
*** The currently working TMP dev version is not planned to push out until January
*** Does {{bug|761723}} have too much code change to back out (it's the potential regressing bug) - engineering volunteer to work on a potential code fix?
** {{bug|588909}} fallout:
*** No topsites broken, but many smaller sites reporting issues with DHTML menus ({{bug|814138}},{{bug|780393}})
*** Broken libraries: certain embeddings of FCKeditor (including some financial institutions mentioned in SUMO feedback), Moodle workaround for UA is now affecting their cross-site SSO system, MNet (namely Mahara) and either requires yet another hack (slippery slope) or a backout of UA changes
*** several smaller data entry and rich editing libraries
*** Question: How important/how much of a win is the UA change that we should continue to take these papercuts or just rip out the change (for the forseeable future, given a lack of TE options)?

==Products/Projects==

===Firefox OS===

*QA
** QA team started last week, began executing Functional testruns
*** Pass: 778, Fail: 158, Invalid: 175
*** This first pass is giving us tons of feedback on testcases that need fixing (outdated steps, need to move to v2)
** Smoketest blockers, still shows 16 open
*** 11-26 smoketests issues:
*** [NEW] [CALL LOG] Outgoing calls are not appearing in call log {{Bug|815063}}
*** [NEW] [BLUETOOTH] Receive images from another device is not working {{Bug|815079}}
*** [NEW] [sms] Compose/share activities are unable to return to calling app once message is sent {{Bug|815093}}
*** [EXISTING] [sms] No sound is played when receiving a text message with vibration and sound enabled {{Bug|811087}}
*** [EXISTING] [camera] Appears a lot of vertical lines on the screen after you take a photo. {{Bug|812208}}
*** [EXISTING] Videos are recorded at 90deg {{Bug|812924}}
*** [EXISTING] [Apps] Unable to install packaged apps when running OOP {{Bug|813468}}
*** [EXISTING] [Contact] Crash after importing around 445 contacts from Facebook {{Bug|813446}}
*** [EXISTING] {{Bug|815274}} - [SMS] - Can't open the SMS application from a newly arrived SMS message in the notification panel
** Gaia Automation work:
*** test stability is being addressed, with a few wins, few setbacks (see Issues/Risks)
*** working on getting Wi-Fi support in, to unblock many other tests
*** added lockscreen tests
*** with a couple soon-to-be merged pulls, we'll have 6/14 tests passing
*** Really close to having an in-tree Contributor document in GitHub (iterating over it, should be ready by next week)
*** Likewise, really close to a review process, with a core group defined
*** still having test-reliability issues (both with Marionette + Gaia itself):
*** bug 814037 - is_displayed struggling with transforms
*** bug 813583 - [B2G] Homescreen iframe not interpreted as visible
*** bug 814151 - Add a "camera-ready" or similar event to help tests ensure Camera's ready for tests to interact with
** A few blockers in progressing more testcases
** Other blockers
*** busted packaged apps
*** 3rd party app work still being discussed
*** Sound testcases late

===Firefox Desktop===
*Social API
** Short term focus is on multi-provider support, should be landing this week.
*New downloads panel
** Uplifted to Aurora for testing. Most blockers are now fixed, but Downloads view for the library remains.
*Per-window PB
** Ship target is Firefox 20
*Snappy
** Big focus is on main thread I/O: Downloads, Form History/Autocomplete, Content Prefs, SessionStore, search service, Thumbnails, places
* Australis
** Windows tabs nearing completion, in final review stages.

===Firefox Mobile===
; Blog Shout Out
* Read Jchen's post about using ADB via an SSH Tunnel: http://www.jnchen.com/blog/2012/11/tunnelling-adb

;Usability & Responsiveness
* Landed in Fx19 {{Bug|808663}} Downscale awesome-bar row Favicons from 32dip to 16dip
* To save memory for ARMv6 devices, {{Bug|792603}} Disable automatic reader mode parsing
* \o/ Patch queue landed in Fx20: {{Bug|783368}} - Add a low res tile cache to TiledThebesLayer
** T-Checkboard has improved but T-Panning regressed, we're evaluating tradeoffs in {{Bug|814864}} Low precision tile rendering takes too much time away from high precision rendering

;Stability Wins
* Fixed for Fx19 {{Bug|802510}} - java.lang.RuntimeException: Buffer not large enough for pixels at android.graphics.Bitmap.copyPixelsFromBuffer(Bitmap.java) especially on Android 4.2
* Fixed for Fx19 {{Bug|812594}} - PushLocalFrame in AutoLocalJNIFrame::Push taking more than 100ms when called from AndroidGeckoLayerClient::ProgressiveUpdateCallback on a Samsung Galaxy Q
* Fixed for Fx20 {{Bug|775142}} - java.util.UnknownFormatConversionException: Conversion: at java.util.Formatter$FormatSpecifierParser.unknownFormatConversionException(Formatter.java) mainly with Japanese and Chinese builds

;Work in Progress
* In the interest of QVGA support for ARMv6, {{Bug|716403}} Request to hide the navigation bar when scrolling down content
* Working on getting WebRTC stood up {{Bug|750869}} - Support WebRTC for Android in our build system

===Metro [Weekly]===

==== Platform ====

* XPC Shell test work continues (only one failing test left!) {{bug|789335}}
* Services code will be moving from app to gre {{bug|815320}}
* New mechanism for defining app-dirs in xpcshell.ini files {{bug|810617}}
* New rev of packager maybe this week
* Migration work from elm to m-c continues to progress

==== Rel-eng ====

* All win64 builders now have the 8.0 SDK installed
* Planning to switch mc to the 8.0 sdk after the next merge on ~2013-01-06 {{bug|774910}}

==== Front-end ====
* Patches in progress to simplify touch input and send touch events to content ({{bug|794621}} and dependencies).
* Starting work on download manager.
* Landing some UX tweaks and bug fixes.

===Stability Report [Weekly]===

* We are doing pretty good stability-wise in recent releases, crash rates are low, most crashes we talk about post-release are third-party issues and not our own code. Keep it up!
* Overview of stability indicators: [http://arewestableyet.com/ arewestableyet.com]
* Looking into how to make this report in this meeting more attractive for attendees - what are you interested in? Please tell stability@m.o or #crashkill

* Trunk stats are still weeding out Win64 GC crashes from a big spike last week ({{bug|813619}}) - but we have '''[https://crash-stats.mozilla.com/topcrasher/by_build_date/Firefox/20.0a1/3/browser top crashes by build date]''' reports that are clean from that!

* Trunk bugs looking for assignees:
** {{bug|805745}} - gfxASurface::Release() is the #2 signature over the last 3 days by build date
** {{bug|815542}} - The spike is Linux only, so not showing up high in overall volume, but it's out #1 crash on Linux in the last 3 days

====Mobile====
[[File:CrashStats_2012-11-27.png]]

* [https://crash-stats.mozilla.com/topcrasher/byversion/FennecAndroid/17.0/3/all 17 Release top crashes]
* [https://crash-stats.mozilla.com/topcrasher/byversion/FennecAndroid/18.0b1/3/all 18.0b1 Beta top crashes]
* [https://crash-stats.mozilla.com/topcrasher/byversion/FennecAndroid/19.0a2/3/all Aurora top crashes]
* [https://crash-stats.mozilla.com/topcrasher/byversion/FennecAndroid/20.0a1/3/all Nightly top crashes]

====B2G====
* We still get too few reports sent in overall, so again, please test what you can on your devices and send reports if you crash!
* {{Bug|811778}} waiting for uplift, will only send reports over wifi.
* We'll need {{bug|814078}} as a followup to make sure we submit all queued/pending reports.
* Also, {{bug|811335}} on the server (stackwalker) side is currently (probably) blocking us from getting better information out of the reports we receive.

===Snappy [Weekly]===
* Vladan Djeric blogged about the [https://blog.mozilla.org/vdjeric/2012/11/25/new-abouttelemetry-page-in-firefox-19/ new about:telemetry page in Firefox 19]

;Content
*Aaron Klotz has made some great progress on prompting for plug-in hangs. See his [http://dblohm7.ca/blog/2012/11/22/plugin-hang-user-interface-for-firefox/ blog post] and check it out in this [http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/aklotz@mozilla.com-192a654f8ad6/try-win32/firefox-19.0a1.en-US.win32.installer.exe try build].

;Admin
* Taras will be out until Jan 1. Contact Vladan Djeric for technical stuff, Dmandelin or JP for everything else.

===Games [1st Tuesday of Month]===

===Mobile Web Compat [2nd Tuesday of Month]===

===Critsmash [3rd Tuesday of Month]===

===Memshrink [4th Tuesday of Month]===

* B2G still a primary focus. It's harder on beta to make significant changes. Cross your fingers.

* The (hopefully) last compartment sharing work for b2g is on central and is ready to land on Aurora once the tree reopens. After that we can land it all and turn it on on beta.

* Lots of work improving coverage of memory reporters, mostly to help B2G.

* We briefly hit zero known leaks in add-ons! A year ago this was our single biggest memory consumption problem, and now it's a minor annoyance. Great stuff.

* MemShrink reports: [https://blog.mozilla.org/nnethercote/2012/10/31/memshrink-progress-week-71-72/ Week 71--72], [https://blog.mozilla.org/nnethercote/2012/11/14/memshrink-progress-week-73-74/ Week 73--74].

==Engineering Metrics==

==Key Issues==
=== Actions Last Week ===
* Mossop to follow up to mailing list about super-review status, update doc and post link
* jduell to update style guide for anon name spaces and mail to list

=== Issues ===
* Namespace style guidelines (khuey)
* Not shipping prefixed APIs on the release channel (henri)
* Future Issues - early discussion on dev-platform, monitoring for discussion if not resolved
** UA String Touch or Tablet
** Win64 builds

=== Actions ===

==Roundtable==

Platform/2012-Q4-Goals

2012-10-29T21:16:23Z

Tglek: /* Perf */

=== General Goals ===

=== GFX ===
* {{ok|Prototype GPU acceleration for Azure/Skia canvas on mobile}}
* {{ok|Prototype GPU acceleration for Azure/Skia canvas on Windows}}
** Determine if this is possible on Windows XP due to context loss etc
* {{ok|Have a clean, dependable, extensible, all-platform, free-of-adverse-cache-effects driver blacklisting solution.}}
* {{ok|Enable Azure for content everywhere}}
* {{ok|OMTC implemented, landed and turned on for Metro product beta}}
* {{ok|Turn on GL layers on Linux}}
* Other important Q4 work
** OMTC on Direct3D 9, OpenGL on Desktop, basic layers
** WebGL test suite compliance
** CSS font spec work
** Convert drawing code to Azure
** Other B2G and mobile work as necessary

=== Layout ===

<onlyinclude>
* {{ok|Readability 2.0}}
** list features
* {{ok|Complete Graphite Fonts Project}} ({{bug|631479}})
** needs final sec-review
* {{ok|Invalidation via DisplayList Analysis ({{bug|539356}})}}
** fix regressions
* {{ok|CSS Flexbox feature/spec}} ({{bug|666041}})
** pref'ed on
* {{ok|Add image visibility API ({{bug|689623}})}}
** r+ and land
* {{ok|CSS}} [[Platform/Features/Vertical_text|Vertical Text]] ({{bug|145503}})
** list sub-bugs for Q4
* {{ok|CSS Pagination}} ({{bug|775628}})
** list sub-bugs for Q4
* {{ok|CSS Variables}} ({{bug|773296}})
* {{ok|Implement the auto value for the HTML dir attribute ({{bug|548206}})}}
** patch needs r+ and landed
* {{ok|CSS 2.1 Test Suite v.2}}
* {{ok|CSS Spec, Unprefixing & Testing}} ({{bug|775235}})
** need sub-bugs
* {{ok|Scoped Style Sheets}} ({{bug|508725}})
** need to specify Q4 to-do list
* {{ok|SVG Text}} ({{bug|655877}})
** need to specify Q4 to-do list
* {{risk|Off Main Thread Animations}} ({{bug|755084}}, {{bug|706179}})
** need a resource to pick up dzbarsky's work
* {{risk|Layout Azure Conversion}} ({{bug|776197}}
** needs staffing
* {{ok|Support for Complex Scripts on Mobile}}
** list actual Q4 scope
** Example bugs: {{bug|619521}}, {{bug|619524}}, {{bug|676068}}, {{bug|631159}}

</onlyinclude>

=== Media ===
* {{ok | WebRTC enabled by default (not behind a pref or config flag) in time for Firefox 20 uplift}}
* {{ok | Land media platform decoding functionality for Android's Gingerbread and Honeycomb OSs in mozilla-central. One target device for each OS will be identified and agreed upon by Media and Android to verify functionality. Success criteria: platform decoders running in Firefox will playback videos at a similar performance metric to the videos playing back in the default browser on the same device.}}
* {{ok | Demo H.264/MP4 support using Windows Media Foundation on Windows 7 or Windows 8 Metro }}

=== DOM ===
* {{done|Make new DOM bindings codegen generate stub class for implementers (in a text file or what not).}}
* {{prev|New DOM bindings for HTMLElement.}}
* {{ok|new binding for HTMLDocument}}
* {{ok|new bindings for 10 HTML element classes}}
* {{ok|make calling new binding callbacks easy and safe}}

=== WebAPI ===
* {{prev|Temporary storage for IndexedDB ({{bug|785884}})}}
* {{prev|Implement unified offline storage quota system, putting IndexedDB and localStorage under this quota system ({{bug|767944}})}}
* {{ok|Assist with reducing memory usage and increasing performance of B2G ({{bug|797189}})}}
* {{ok|Make B2G v1 a high quality release}}
* {{ok|USBCode in DOM key events}}
* {{ok|Number form support for desktop}}
* {{ok|Develop and submit to W3C a proposal for "fixing appcache"}}

=== JS ===

*{{prev|Incremental sweeping by compartment}} [{{bug|790338}}]
*{{prev|Generational GC: get automated safety checks running and green}}
*{{ok|Manage memory by zones}} (to reduce memory usage) [{{bug|759585}}]
*{{prev|Finish properties/elements split}}

=== Accessibility ===
* {{ok|FF OS for users who are blind (land accessfu + TTS on trunk pref'ed on).}}
* {{ok|AccessFu/jsat test coverage.}}
* {{risk|Initial UIA implementation (Narrator support).}}

=== Perf ===
* {{prev|{{nbug|736144}}: Async local storage via blocking pageload}}
* {{prev|{{nbug|662397}}: Reorder xul.dll on windows to speed up startup}}
* {{prev|{{nbug|662444}}: call exit(0) on shutdown}}
* {{prev|{{nbug|661881}}: Bundle about-telemetry extension to ship with Firefox}}
* {{prev|{{nbug|769241}}: Make libunwind work on ARM}}
* {{prev|Prevent, to a reasonable extent, background tabs from starving the main thread}}

=== Networking ===

* {{ok|Resolve all networking basecamp blockers that are marked as blocking before November 1st.}}
** This work will be done primarily by Jason Duell, Honza Bambas, and Brian Smith.
* {{ok| {{nbug|713933}} Enable TLS false start for NPN/SPDY in mozilla-central.}}
** Patrick McManus will lead this effort.
* {{ok|Remove all potentially blocking disk cache API usage in nsHTTPChannel and nsWYCIWYGChannel, on mozilla-central.}}
** Nick Hurley and Michal Novotny will lead this effort.
* {{ok|{{nbug|702122}}: Have full adaptive DASH for WebM video enabled by default on Aurora.}}
** Steve Workman will lead this effort.
* {{ok|Resolve all networking security bugs that received a designation of sg:moderate or higher more than six weeks ago. This is a permanent goal for the group.}}

=== Plugins ===

=== Mobile ===

* {{ok|Resolve all mobile security bugs that received a designation of sg:moderate or higher more than six weeks ago. This is a permanent goal for the group.}}
* {{ok|Support shipping ARMv6}}
* {{ok|Ship H.264 on ICS & JB}} [{{bug|787227}}]
* {{ok|Implement support for H.264 on GB on trunk}} [{{bug|787228}}]
* {{ok|Investigate support for Java addons [{{bug|799631}}]}}
* {{ok|Determine if we can be competitive with the stock browser on 256Mb devices [{{bug|792131}}]}}
* {{ok|Ship safe browsing to Beta}}
* {{ok|Support shipping fonts in the APK in Beta}}
* {{ok|Support shipping to an alternative app store with improved updater}}
* {{ok|Make full screen video experience pleasant for the user [{{bug|800673}}]}}

Dependencies
* QA for shipping ARMv6
* Media team for H.264 support

=== B2G ===

=== Research ===

Platform/2012-Q3-Goals

2012-10-29T21:15:38Z

Tglek: /* Perf */

=== General Goals ===

=== GFX ===
<onlyinclude>
* {{miss|stop using Java screenshotting in Fennec for Android}}
** we agree that we want to do this, but we have concerns that the pieces will be there, but we'll still be unable to turn off java screenshotting because of performance problems.
** alternative: "implement multi-resolution tiles"
* {{done|complete asynchronous panning and zooming with OMTC for B2G/gonk}}
* {{done|Ship OMTC on B2G/gonk for basecamp}}
* {{done|Be able to use Skia for canvas drawing on non-accelerated Windows computers}}
** gw280 to gather skia software-only performance numbers by Tuesday 3 July to help make a decision
** carryover - do we care at this point, given the below item?
* {{miss|Be able to use Skia for content drawing on Android}}
* {{done|Remove the non-Azure canvas implementation}}
** Use the Cairo Azure backend everywhere there isn't a more specific backend
* {{miss|OMTC cross-platform refactor}}
** Risky!
** Not back-outable. No OMTC or "New OMTC" only.
** Needed so we can enable OMTC everywhere.
* {{miss|Have a clean, dependable, extensible, all-platform, free-of-adverse-cache-effects driver blacklisting solution.}}
** {{bug|668008}} - tracking bug
* Other important Q3 work
** WebGL work. https://etherpad.mozilla.org/webgl-q3-goals
** Enable Azure for content everywhere
** Snappy work
** Other B2G work as required
** Moving other pieces of work off the main thread
** BugKill - ???
** Increase number of regular contributors
</onlyinclude>

=== Layout ===

<onlyinclude>
* {{prev|Readability 2.0}}
** {{done|High-DPI on MacOSX Retina}}
** {{done|Reflow-on-Zoom landed}}
** {{done|APK Fonts on Android landed}}
* {{prev|Complete Graphite Fonts Project}} ({{bug|631479}})
** {{done|Updated Libraries from SIL}}
* {{prev|Invalidation via DisplayList Analysis ({{bug|539356}})}}
** {{done|Landed in Q3}}
* {{prev|CSS Flexbox feature/spec}} ({{bug|666041}})
** {{done|Landed in Q3}}
* {{prev|Add image visibility API ({{bug|689623}})}}
** {{done|Try Server Builds for testing}}
* {{risk|Continue View class removal ({{bug|337801}})}}
* {{prev|CSS}} [[Platform/Features/Vertical_text|Vertical Text]] ({{bug|145503}})
** {{done|First Prototypes}}
* {{prev|CSS Pagination}} ({{bug|775628}})
** {{done|({{bug|685012}}) in r?}}
* {{prev|CSS Variables}} ({{bug|773296}})
* {{prev|Implement the auto value for the HTML dir attribute ({{bug|548206}})}}
** {{done|patch needs r?}}
* {{prev|CSS 2.1 Test Suite v.2}}
* {{prev|CSS Spec, Unprefixing & Testing}} ({{bug|775235}})
* {{done|Conditional Rules feature & spec}} ({{bug|649740}})
* {{prev|Scoped Style Sheets}} ({{bug|508725}})
* {{prev|SVG Text}} ({{bug|655877}})
** {{done|lots of dependent bugs closed/fixed}}
* {{prev|Off Main Thread Animations}} ({{bug|755084}}, {{bug|706179}})
** {{done|lots of progress for B2G}}
* {{risk|Layout Azure Conversion}} ({{bug|776197}} - needs staffing)
* {{prev|Support for Complex Scripts on Mobile}}
** Example bugs: {{bug|619521}}, {{bug|619524}}, {{bug|676068}}, {{bug|631159}}
* Specs:
** Web Animations (FPWD)
** CSS Flexbox (CR)
** CSS Cascade L3 (spec scoped style)
** CSS Conditional L3 (CR)
** CSS Text L3 (LCWD)
** Selectors L4
</onlyinclude>

=== Media ===
* {{done|WebRTC landed for desktop in mozilla-central behind a pref in time for Firefox 18. (This includes both PeerConnection and full getUserMedia, but not UI.)
** We wound up landing DataChannels and basic UI as well}}
* {{done|Platform decoders running on the Otoro device will playback videos at a similar performance metric to the equivalent android player}}
* {{done|For a specific list of Android phones (ICS/JB devices), platform decoders running in Firefox will playback videos at a similar performance metric to the videos playing back in the default browser on the same phone}}
* {{done|Publish (to one or more of our newsgroups) a plan for supporting the Web Audio API. (We're still hoping to make improvements to the spec.)}}

Mission Note: The media team will support B2G's efforts to ship B2G version 1 above any goals for this quarter.

=== DOM ===
* {{done|Stop leaks by adding purple buffer support for cycle collected non-nsISupports objects.}}
* {{deferred|Speed up cycle collection by representing a DOM tree as a single node in the cycle collector graph.}}
** A good bit of investigation and experimentation was done here, but it's a harder problem than we believed it would be, and security bug fixing took precedence over completing this.
* {{miss|New DOM bindings for a HTML Element.}}
** 95% done, but didn't get reviews in time for the end of the quarter, and even if we had gotten that, we would not land this a week before branching for 19. We'll finish this up early in Q4.

=== WebAPI ===
* {{done|Implement cookie-jars for cookies, IndexedDB, localStorage, permission manager and appcache ({{bug|756644}})}}
* {{done|Implement app:// protocol (part of implementing trusted apps, {{bug|769350}})}}
* {{done|Finalize multiprocess blob support for IndexedDB, DeviceStorage and Message Manager ({{bug|759427}})}}
* {{deferred|Temporary storage for IndexedDB ({{bug|785884}})}}
* {{deferred|Implement unified offline storage quota system, putting IndexedDB and localStorage under this quota system ({{bug|767944}})}}
* {{done|Zip file contents support using blobs ({{bug|772434}})}}
* {{done|DeviceStorage improvements (expose available (free) size for each device storage, storage[x].removable ({{bug|765498}})}}
* {{deferred|FileHandle [[WebAPI/FileHandleAPI#ToDo | improvements]]}}
* {{done|DeviceStorage onchange notifications ({{bug|763976}})}}
* {{deferred|DeviceStorage editable features ({{bug|752724}})}}
* {{done|String encoding/decoding API ({{bug|764234}})}} (completed with community help)

=== JS ===

*{{done|Land IonMonkey}}
*{{miss|Incremental sweeping by compartment}} [{{bug|780960}}] - not done but is very close to done.
*{{miss|Generational GC: get automated safety checks running and green}}
*{{miss|Finish properties/elements split}}
*{{done|ES6: direct proxies}} [{{bug|703537}}]
*{{miss|ES6: modules}} [{{bug|568953}}]

=== Accessibility ===
* {{done| Preliminary AccessFu support in B2G.}}
** not landed (targeting v2)
* {{deferred| TTS Web API}}
** {{done| Implementation of TTS for b2g. Focus here morphed to just getting what we need for the B2G goal, and helping with B2G blockers (crash reporting work).}}
* {{miss| Performance: add two additional measures of a11y's effect on perf. Additionally, show measurable perf increase of at least 10% on two measures.}}
** Minor performance work landed but there was no precise measuring.

=== Perf ===
* {{prev|{{nbug|736144}}: Async local storage via blocking pageload}}
* {{done|{{nbug|563742}}: provide js file api (in workers) for all supported platforms.}}
* {{prev|{{nbug|662397}}: Reorder xul.dll on windows to speed up startup}}
* {{prev|{{nbug|662444}}: call exit(0) on shutdown}}
* {{prev|{{nbug|661881}}: Bundle about-telemetry extension to ship with Firefox}}
* {{ok|{{nbug|769241}}: Make libunwind work on ARM}}
* {{prev|Prevent, to a reasonable extent, background tabs from starving the main thread}}
* {{done|{{nbug|770317}}: Track disk + network by thread on testing infrastructure}}

=== Networking ===

* {{miss|Remove all synchronous disk cache APIs, or at least disable their use on the main thread, on mozilla-central.}}
** Nick Hurley and Michal Novotny will lead this effort.
** We just didn't get this done, lots of complications.
* {{done|{{nbug|766973}}: Don't allow synchronous DNS resolution from the main thread.}}
** Josh Aas will lead this effort.
** We will not be able to stop the Java plugin from being able to do proxy resolution that may result in DNS resolution on the main thread, but all other cases will be taken care of and a plan is in place for Java.
* {{miss|{{nbug|105843}}: Land code to not necessarily delete the cache after unclean shut-down on mozilla-central.}}
** Nick Hurley and Michal Novotny will lead this effort.
** Brian Bondy is helping with this.
** We just won't get this done in time.
* {{done|Have Stone Ridge reliably reporting basic performance results to graph server for all three top-tier platforms.}}
** Nick Hurley will lead this effort.
* {{miss|{{nbug|768705}}, {{nbug|704447}}, {{nbug|766817}}: Fix some remaining significant WebSockets bugs.}}
** Jason Duell will lead this effort.
** Priority given to B2G-specific efforts that came up, this goal was abandoned.
* {{done|{{nbug|702122}}: Land a DASH for WebM implementation in mozilla-central.}}
** Steve Workman will lead this effort.
** Landed support for loading video via DASH and video switching. Haven't actually landed an adaptive algorithm yet.
* {{done|{{nbug|737470}}: Ship SPDY v3 on by default.}}
** Patrick McManus will lead this effort.
* {{miss|Resolve all networking security bugs that received a designation of sg:moderate or higher more than six weeks ago. This is a permanent goal for the group.}}
** Brian Smith will lead this effort.
** Priority given to B2G-specific efforts that came up, this goal was abandoned. We only have one applicable bug remaining, however.

=== Plugins ===

=== Mobile ===
* {{done|{{nbug|741808}} turn on safe browsing}}
* {{done|{{nbug|786380}} improve updater}}
* {{done|{{nbug|759945}} h264 support for ICS/JB}}
* {{done|{{nbug|687267}} Flash support on Honeycomb}}

=== B2G ===

=== Research ===

Performance/Snappy

2012-10-23T23:50:36Z

Tglek: /* Bug Tracking */

=Project "Snappy"=

==Goal==
Project Snappy will make improvements to Firefox desktop responsiveness by completing smaller, isolated pieces of work.

This project is a responsiveness equivalent of [[Performance/MemShrink]].

===Scope===
* Code that impacts the responsiveness of the Firefox desktop browser
* Out of scope
** Major architectural changes such as electrolysis (e10s) and supersnappy
** Firefox mobile, apps WebRT, B2G
** Development of performance benchmarks
** UI changes for perceived performance

==Projects==

* Temporary & Severe Browser Hangs
* Content Responsiveness (intermittent hangs)
* [[Performance/Snappy/Startup&Shutdown | Start-up & Shutdown]]
* Tools
* Add-on and Web page introduced slowness
* Page load
* [[Performance/Snappy/Tabstrip | Tabstrip]]

==Communication==
{| class="wikitable fullwidth-table"
! Communication Type !! Mechanism !! Audience
|-
| Announcements || [https://lists.mozilla.org/listinfo/dev-platform dev-platform] and [https://lists.mozilla.org/listinfo/dev-planning dev-planning] lists || all
|-
| General discussion || [https://lists.mozilla.org/listinfo/dev-platform dev-platform] list || devs
|-
| Meetings || Alternate Thursdays, 11am PT
{{conf|95346}}
* Vidyo: PB&J
* IRC: [irc://irc.mozilla.org/perf #perf]
* Etherpad: [https://etherpad.mozilla.org/snappy snappy] (copied to wiki after the meeting)
|| all
|-
| Meeting summaries || [[Performance/Snappy | this wiki]] and [https://blog.mozilla.org/tglek/ Taras Glek's blog] || all
|}

===Press===
* May 14, 2012 - [http://www.pcworld.com/article/255569/firefox_13_gets_a_triple_shot_of_speed_through_mozillas_project_snappy.html PCWorld: Firefox 13 Gets a Triple Shot of Speed Through Mozilla's 'Project Snappy']
* May 11, 2012 - [http://www.webpronews.com/mozilla-makes-firefox-13-super-speedy-2012-05 WebProNews: Mozilla Makes Firefox 13 Super Speedy]

===Blog Posts===
* Sep 22, 2012 - [http://msujaws.wordpress.com/2012/09/22/firefox-performancesnappy-work-week-warsaw-edition/ Jared Wein: Firefox Performance/Snappy Work Week: Warsaw edition]
* Sep 18, 2012 - [https://blog.mozilla.org/tglek/2012/09/18/snappy-in-warsaw-pierogy-fueled-hackfest/ Taras Glek: Snappy in Warsaw: pierogy-fueled hackfest]
* May 11, 2012 - [https://hacks.mozilla.org/2012/05/getting-snappy-performance-optimisations-in-firefox-13/ Lawrence Mandel: Getting snappy – performance optimizations in Firefox 13]
* Mar 14, 2012 - [http://autonome.wordpress.com/2012/03/14/firefox-11-is-smaller-and-faster/ Dietrich Ayala: Firefox 11 is Smaller and Faster]
* Feb 8, 2012 - [http://www.brianbondy.com/blog/id/127/ Brian Bondy: Snappy optimizations for faster Firefox startup]

=== Minutes and Progress Reports ===
{| class="wikitable collapsible" style="width: 100%"
! style="background-color: rgb(221, 221, 221);" colspan="2" | 2012
|-
|
* Thursday, October 18, 2012 - [[Performance/Snappy/2012-10-18| Minutes]], [https://blog.mozilla.org/tglek/2012/10/18/snappy-41/ Snappy #41]
* Thursday, October 11, 2012 - [[Performance/Snappy/2012-10-11| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/10/15/snappy-40-faster-tabswitching-startup-analysis/ Snappy #40: Faster tabswitching, startup analysis]
* Thursday, October 4, 2012 - [[Performance/Snappy/2012-10-04| Minutes]], [https://blog.mozilla.org/tglek/2012/10/04/snappy-39/ Snappy #39]
* Thursday, September 27, 2012 - [[Performance/Snappy/2012-09-27| Status (no meeting)]]
* Thursday, September 20, 2012 - [[Performance/Snappy/2012-09-20| Minutes]]
* Snappy work week Sep 10-14. You may want to read [https://blog.mozilla.org/tglek/2012/09/18/snappy-in-warsaw-pierogy-fueled-hackfest/ Taras' summary]
* Thursday, September 6, 2012 - [[Performance/Snappy/2012-09-06| Minutes]]
* Thursday, August 30, 2012 - [https://blog.mozilla.org/tglek/2012/09/04/snappy-38/ Snappy #38]
* Thursday, August 23, 2012 - [[Performance/Snappy/2012-08-23| Minutes]], Progress report: [https://blog.mozilla.org/tglek/2012/08/23/snappy-37/ Snappy #37]
* Thursday, August 16, 2012 - [[Performance/Snappy/2012-08-16| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/08/16/snappy-36/ Snappy #36]
* Thursday, August 9, 2012 - [[Performance/Snappy/2012-08-09| Minutes]], [https://blog.mozilla.org/tglek/2012/08/14/snappy-for-aug-9/ Snappy for Aug 9]
* Thursday, August 2, 2012 - [[Performance/Snappy/2012-08-02| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/08/06/snappy-aug-2/ Snappy, Aug 2]
* Thursday, July 26, 2012 - [[Performance/Snappy/2012-07-26| Minutes]], [https://blog.mozilla.org/tglek/2012/07/26/snappy-july-26-go-try-the-gecko-profiler/ Snappy, July 26: Go Try The Gecko Profiler!]
* Thursday, July 19, 2012 - [[Performance/Snappy/2012-07-19| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/07/23/snappy-july-19-telemetry-experiments/ Snappy, July 19: Telemetry Experiments]
* Thursday, July 12, 2012 - [[Performance/Snappy/2012-07-12| Minutes]]
* Thursday, July 5, 2012 - [[Performance/Snappy/2012-07-05| Status (no meeting)]]
* Thursday, June 28, 2012 - [[Performance/Snappy/2012-06-28| Minutes]]
* Thursday, June 21, 2012 - [[Performance/Snappy/2012-06-21| Status (no meeting)]]
* Thursday, June 14, 2012 - [[Performance/Snappy/2012-06-14| Minutes]]
* Thursday, June 7, 2012 - [[Performance/Snappy/2012-06-07| Status (no meeting)]]
* Thursday, May 31, 2012 - [[Performance/Snappy/2012-05-31| Minutes]]
* Thursday, May 24, 2012 - [[Performance/Snappy/2012-05-24| Status (no meeting)]]
* Thursday, May 17, 2012 - [[Performance/Snappy/2012-05-17| Minutes]]
* Thursday, May 10, 2012 - [[Performance/Snappy/2012-05-10| Status (no meeting)]]
* Thursday, May 3, 2012 - [[Performance/Snappy/2012-05-03| Minutes]]
* Thursday, April 26, 2012 - [[Performance/Snappy/2012-04-26| Minutes]]
* Thursday, April 19, 2012 - [[Performance/Snappy/2012-04-19| Minutes]]
* Thursday, April 12, 2012 - [[Performance/Snappy/2012-04-12| Minutes]]
* Thursday, April 5, 2012 - [[Performance/Snappy/2012-04-05| Minutes]]
* Thursday, March 29, 2012 - [[Performance/Snappy/2012-03-29| Minutes]]
* Thursday, March 22, 2012 - [[Performance/Snappy/2012-03-22| Minutes]]
* Thursday, March 15, 2012 - [[Performance/Snappy/2012-03-15| Minutes]]
* Thursday, March 8, 2012 - [[Performance/Snappy/2012-03-08| Minutes]]
* Thursday, March 1, 2012 - [[Performance/Snappy/2012-03-01| Minutes]]
* Thursday, February 23, 2012 - [[Performance/Snappy/2012-02-23| Minutes]]
* Thursday, February 16, 2012 - [[Performance/Snappy/2012-02-16| Minutes]]
* Thursday, February 9, 2012 - [[Performance/Snappy/2012-02-09| Minutes]]
* Thursday, February 2, 2012 meeting cancelled due to travel
* Thursday, January 26, 2012 - [[Performance/Snappy/2012-01-26| Minutes]]
* Thursday, January 19, 2012 - [[Performance/Snappy/2012-01-19| Minutes]]
* Thursday, January 12, 2012 - [[Performance/Snappy/2012-01-12| Minutes]]
* Thursday, January 5, 2012 meeting cancelled due to holidays
|}

{| class="wikitable collapsible collapsed" style="width: 100%"
! style="background-color: rgb(221, 221, 221);" | 2011
|-
|
* Thursday, December 29, 2011 - meeting cancelled due to holidays
* Thursday, December 22, 2011 - [[Performance/Snappy/2011-12-22| Minutes]], [https://blog.mozilla.org/tglek/2011/12/22/snappy-dec-22/ Progress report]
* Thursday, December 15, 2011 - [[Performance/Snappy/2011-12-15| Minutes]], [https://blog.mozilla.org/tglek/2011/12/16/snappy-update-for-dec-15/ Progress report]
* Thursday, December 8, 2011 - [[Performance/Snappy/2011-12-08| Minutes]], [https://blog.mozilla.org/tglek/2011/12/13/snappy-summary-for-dec-8/ Progress report]
* Thursday, December 1, 2011 - [[Performance/Snappy/2011-12-01| Minutes]], [https://blog.mozilla.org/tglek/2011/12/01/introducing-project-snappy/ Progress report]
* Friday, November 18, 2011 - [[Performance/Snappy/2011-11-18| Minutes]]
|}

== Bug Tracking ==
Bugs tracked by the Snappy project are prioritized by adding one of [Snappy:P1], [Snappy:P2], or [Snappy:P3] to the whiteboard. Use whiteboard [Snappy] tag to nominate bugs.

* [[Performance/Snappy/Dashboard | Snappy dashboard]] - shows recent Snappy changes

* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=Snappy%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized Snappy bugs]. These are triaged occasionally.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P1 bugs]. These are discussed regularly in meetings.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p1]",
"x_axis_field": "status"
}
</bugzilla>
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P2 bugs]. These are discussed occasionally in meetings. P2 is the default priority.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p2]",
"x_axis_field": "status"
}
</bugzilla>
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P3 bugs]. These are discussed rarely in meetings.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p3]",
"x_axis_field": "status"
}
</bugzilla>

Some other interesting bugs/lists.
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp;query_format=advanced;status_whiteboard=telemetry-needed;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;resolution=---;resolution=DUPLICATE;list_id=2435528 bugs that need telemetry probes implemented]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=698500 visible unresponsiveness metabug]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=699820 sync storage main-thread-io metabug]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=698919 cycle collector responsiveness metabug]

== Goals ==
* 50ms responsiveness when typing in a textbox ({{bug|703668}})
* 60fps animations on UI operations, i.e. closing tabs ({{bug|702509}})
* track above via telemetry

more goal thinking [https://etherpad.mozilla.org/desktop-responsiveness responsiveness goals]

My (Asa) proposed core measures:
* Startup on par with IE and Chrome
* 50ms responsiveness for all non-animated primary UI interactions (list TBD)
* "Above the fold" uncached page load on par with IE and Chrome
* Back/Forward navigation as fast as Opera.

== Snappy Addons ==
If you discover an addon that may be causing responsiveness issues
# Capture a profile of the jank with [https://developer.mozilla.org/en/Performance/Profiling_with_the_Built-in_Profiler SPS Profiler].
# File a bug under Tech Evangelism/Add-ons. Describe the issue, steps to reproduce. [https://bugzilla.mozilla.org/enter_bug.cgi?product=Tech%20Evangelism bugzilla link].
# Upload your SPS profile + post link in bug
# Contact addon author. Authors are usually happy to make their addons more performant.
# If the developer isn't responsive we will get AMO team involved to consider delisting the offending addon version.

See {{bug|777397}} for an example of above process.

== Wins ==
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy",
"resolution": "fixed",
"x_axis_field": "target_milestone"
}
</bugzilla>
;Firefox 16
* [http://bit.ly/LOFsXE All fixed bugs]

;Firefox 15
* Start-up optimization - remove prefetch files (backed out in Firefox 16)
* No more font enumeration
* [http://mzl.la/Iwq9xT All fixed bugs]

;Firefox 14
* IE profile migrator rewrite
* Async favicons
* Slow SQL Telemetry
* [http://bit.ly/GGBpJ5 All fixed bugs]

;Firefox 13
* Tabs on demand
* CC/GC pause reductions
* Start-up optimizations
* Front-end Telemetry
* [http://j.mp/wXnf5x All fixed bugs]

;Firefox 12
* <nowiki>about:jank</nowiki>
* Inline autocomplete
* Slow SQL Telemetry
* [http://j.mp/AhayWF All fixed bugs]

;Firefox 11
* [http://j.mp/ypxFzj All fixed bugs]

==Snappy TODO==
===Firefox===
* {{nbug|712478}}: interactivity heuristics (to reduce jank while playing video, scrolling)
* {{nbug|650968}}: fix personas (startup slowdown, possibly other jank)
* {{nbug|729330}}: fix addon manager (startup slowdown)
* {{nbug|699820}}: fix remaining main thread sqlite (significant source of jank)
* {{nbug|743069}}: fix our tab bar responsiveness (switching tabs, tab animations, etc. We lag other browsers in this area)

===Platform===
* {{nbug|627635}} Make localstorage async (Don't allow web pages to jank us)
* {{nbug|692255}} get rid of windows prefetch (so we can enable library reordering for a big startup win)
* {{nbug|722243}} cancellable sql queries (Cancel background queries to improve interactive performance)
* {{nbug|692557}} Fix GFX acceleration lag (Currently GFX acceleration can result in a severe slowdown over non-accelerated case)
* {{nbug|705594}} Font enumeration lag (This can add minutes of lag on some windows/mac machines)
* OFTC on desktop (Needed for smooth animations)
* Necko improvements (cache, connection pools, etc)

== Current Infrastructure ==

* Event Tracing: http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/EventTracer.cpp#38
** Fires events at main thread event loop from background thread, measures response time
** Produces output of the form MOZ_EVENT_TRACE sample <timestamp> <duration>
** Being rolled out as part of Talos Tp5 {{bug|631571}}, will produce a responsiveness metric for "responsiveness while running Tp5"
* Peptest: {{bug|674606}}
** Responsiveness regression test harness
** Allows devs to write tests that perform various actions and tests responsiveness using EventTracer
** [https://wiki.mozilla.org/Auto-tools/Projects/peptest Project page] contains everything you need to know
** Run 'make peptest' from your object directory to run the tests (see [https://wiki.mozilla.org/Auto-tools/Projects/peptest#Running_Tests running tests] for command line instructions)

* [http://people.mozilla.org/~xstevens/telemetry/telemetry-histogram-aggregates-20111103-20111116.txt Telemetry dump for ff11 from November 17]

* [http://graphs-new.mozilla.org/graph.html#tests=[[130,1,1],[130,1,12],[130,1,13],[130,1,21],[130,1,22]]&sel=1322091900253,1322167344714&displayrange=7&datatype=running Talos responsiveness tests] - for details see {{bug|631571}}

== Additional Infrastructure ==

* Have metrics do continuous trend analysis to see if our telemetry stats are getting better over time.

== arewesnappyyet.com ==
* This should use our telemetry data to tell us whether we are meeting our expectations and how far off we are
* {{bug|703669}} tracks the implementation of this site

== Testing ==
Placeholder section for QA and testing.

; [https://wiki.mozilla.org/Performance/Snappy/Testing:Sprint_1#Phase_1:_UI_Interactions Sprint 1 Phase 1]

; Takeaways from [https://wiki.mozilla.org/Performance/Snappy/2011-12-01 2011-12-01 Meeting]
* Goal of Project:
** Big hangs (~30s) are a problem, but small hangs (~5s) may be a bigger win
** Identify where we hang, why we hang, and plug those holes
* Things on Devs Radar:
** Meta Bugs: visible responsiveness ({{bug|698500}}), sync storage ({{bug|699820}}), cycle collector ({{bug|698919}})
** Animation hangs are most visible
** Windows, Mac, Linux are all unique -- Windows is P1 for the project
** [http://benoitgirard.wordpress.com/2011/11/08/firefox-built-in-profiler/ Firefox Built-in Profiler]: late Q1'2012
** Chromhang Bugs: {{bug|705761}}, {{bug|705287}}, {{bug|705258}}, {{bug|705594}}, {{bug|704933}}, {{bug|699051}}
** Small Jank: using ''refresh driver'' to measure refresh-rate performance (ie. throbber, progress bar, etc -- Asa has more ideas)
** Measuring Snappiness: [http://j.mp/rPGyFb Telemetry use cases]
** Metrics/Automation: for trend-analysis, finding regressions with Telemetry
* Where does QA fit in?
** P1: [https://etherpad.mozilla.org/desktop-responsiveness Desktop responsiveness] competitive testing
** many of these problems need stack traces (something Telemetry cannot do)
** crowd-sourcing use cases where we are less Snappy
** A-B testing with other browsers (versions of Firefox, competitor browsers)
** Ownership of Snappy P1s
** Caching issues should have patches landed in the next week or so

Performance/Snappy

2012-10-23T23:49:42Z

Tglek: /* Bug Tracking */

=Project "Snappy"=

==Goal==
Project Snappy will make improvements to Firefox desktop responsiveness by completing smaller, isolated pieces of work.

This project is a responsiveness equivalent of [[Performance/MemShrink]].

===Scope===
* Code that impacts the responsiveness of the Firefox desktop browser
* Out of scope
** Major architectural changes such as electrolysis (e10s) and supersnappy
** Firefox mobile, apps WebRT, B2G
** Development of performance benchmarks
** UI changes for perceived performance

==Projects==

* Temporary & Severe Browser Hangs
* Content Responsiveness (intermittent hangs)
* [[Performance/Snappy/Startup&Shutdown | Start-up & Shutdown]]
* Tools
* Add-on and Web page introduced slowness
* Page load
* [[Performance/Snappy/Tabstrip | Tabstrip]]

==Communication==
{| class="wikitable fullwidth-table"
! Communication Type !! Mechanism !! Audience
|-
| Announcements || [https://lists.mozilla.org/listinfo/dev-platform dev-platform] and [https://lists.mozilla.org/listinfo/dev-planning dev-planning] lists || all
|-
| General discussion || [https://lists.mozilla.org/listinfo/dev-platform dev-platform] list || devs
|-
| Meetings || Alternate Thursdays, 11am PT
{{conf|95346}}
* Vidyo: PB&J
* IRC: [irc://irc.mozilla.org/perf #perf]
* Etherpad: [https://etherpad.mozilla.org/snappy snappy] (copied to wiki after the meeting)
|| all
|-
| Meeting summaries || [[Performance/Snappy | this wiki]] and [https://blog.mozilla.org/tglek/ Taras Glek's blog] || all
|}

===Press===
* May 14, 2012 - [http://www.pcworld.com/article/255569/firefox_13_gets_a_triple_shot_of_speed_through_mozillas_project_snappy.html PCWorld: Firefox 13 Gets a Triple Shot of Speed Through Mozilla's 'Project Snappy']
* May 11, 2012 - [http://www.webpronews.com/mozilla-makes-firefox-13-super-speedy-2012-05 WebProNews: Mozilla Makes Firefox 13 Super Speedy]

===Blog Posts===
* Sep 22, 2012 - [http://msujaws.wordpress.com/2012/09/22/firefox-performancesnappy-work-week-warsaw-edition/ Jared Wein: Firefox Performance/Snappy Work Week: Warsaw edition]
* Sep 18, 2012 - [https://blog.mozilla.org/tglek/2012/09/18/snappy-in-warsaw-pierogy-fueled-hackfest/ Taras Glek: Snappy in Warsaw: pierogy-fueled hackfest]
* May 11, 2012 - [https://hacks.mozilla.org/2012/05/getting-snappy-performance-optimisations-in-firefox-13/ Lawrence Mandel: Getting snappy – performance optimizations in Firefox 13]
* Mar 14, 2012 - [http://autonome.wordpress.com/2012/03/14/firefox-11-is-smaller-and-faster/ Dietrich Ayala: Firefox 11 is Smaller and Faster]
* Feb 8, 2012 - [http://www.brianbondy.com/blog/id/127/ Brian Bondy: Snappy optimizations for faster Firefox startup]

=== Minutes and Progress Reports ===
{| class="wikitable collapsible" style="width: 100%"
! style="background-color: rgb(221, 221, 221);" colspan="2" | 2012
|-
|
* Thursday, October 18, 2012 - [[Performance/Snappy/2012-10-18| Minutes]], [https://blog.mozilla.org/tglek/2012/10/18/snappy-41/ Snappy #41]
* Thursday, October 11, 2012 - [[Performance/Snappy/2012-10-11| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/10/15/snappy-40-faster-tabswitching-startup-analysis/ Snappy #40: Faster tabswitching, startup analysis]
* Thursday, October 4, 2012 - [[Performance/Snappy/2012-10-04| Minutes]], [https://blog.mozilla.org/tglek/2012/10/04/snappy-39/ Snappy #39]
* Thursday, September 27, 2012 - [[Performance/Snappy/2012-09-27| Status (no meeting)]]
* Thursday, September 20, 2012 - [[Performance/Snappy/2012-09-20| Minutes]]
* Snappy work week Sep 10-14. You may want to read [https://blog.mozilla.org/tglek/2012/09/18/snappy-in-warsaw-pierogy-fueled-hackfest/ Taras' summary]
* Thursday, September 6, 2012 - [[Performance/Snappy/2012-09-06| Minutes]]
* Thursday, August 30, 2012 - [https://blog.mozilla.org/tglek/2012/09/04/snappy-38/ Snappy #38]
* Thursday, August 23, 2012 - [[Performance/Snappy/2012-08-23| Minutes]], Progress report: [https://blog.mozilla.org/tglek/2012/08/23/snappy-37/ Snappy #37]
* Thursday, August 16, 2012 - [[Performance/Snappy/2012-08-16| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/08/16/snappy-36/ Snappy #36]
* Thursday, August 9, 2012 - [[Performance/Snappy/2012-08-09| Minutes]], [https://blog.mozilla.org/tglek/2012/08/14/snappy-for-aug-9/ Snappy for Aug 9]
* Thursday, August 2, 2012 - [[Performance/Snappy/2012-08-02| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/08/06/snappy-aug-2/ Snappy, Aug 2]
* Thursday, July 26, 2012 - [[Performance/Snappy/2012-07-26| Minutes]], [https://blog.mozilla.org/tglek/2012/07/26/snappy-july-26-go-try-the-gecko-profiler/ Snappy, July 26: Go Try The Gecko Profiler!]
* Thursday, July 19, 2012 - [[Performance/Snappy/2012-07-19| Status (no meeting)]], [https://blog.mozilla.org/tglek/2012/07/23/snappy-july-19-telemetry-experiments/ Snappy, July 19: Telemetry Experiments]
* Thursday, July 12, 2012 - [[Performance/Snappy/2012-07-12| Minutes]]
* Thursday, July 5, 2012 - [[Performance/Snappy/2012-07-05| Status (no meeting)]]
* Thursday, June 28, 2012 - [[Performance/Snappy/2012-06-28| Minutes]]
* Thursday, June 21, 2012 - [[Performance/Snappy/2012-06-21| Status (no meeting)]]
* Thursday, June 14, 2012 - [[Performance/Snappy/2012-06-14| Minutes]]
* Thursday, June 7, 2012 - [[Performance/Snappy/2012-06-07| Status (no meeting)]]
* Thursday, May 31, 2012 - [[Performance/Snappy/2012-05-31| Minutes]]
* Thursday, May 24, 2012 - [[Performance/Snappy/2012-05-24| Status (no meeting)]]
* Thursday, May 17, 2012 - [[Performance/Snappy/2012-05-17| Minutes]]
* Thursday, May 10, 2012 - [[Performance/Snappy/2012-05-10| Status (no meeting)]]
* Thursday, May 3, 2012 - [[Performance/Snappy/2012-05-03| Minutes]]
* Thursday, April 26, 2012 - [[Performance/Snappy/2012-04-26| Minutes]]
* Thursday, April 19, 2012 - [[Performance/Snappy/2012-04-19| Minutes]]
* Thursday, April 12, 2012 - [[Performance/Snappy/2012-04-12| Minutes]]
* Thursday, April 5, 2012 - [[Performance/Snappy/2012-04-05| Minutes]]
* Thursday, March 29, 2012 - [[Performance/Snappy/2012-03-29| Minutes]]
* Thursday, March 22, 2012 - [[Performance/Snappy/2012-03-22| Minutes]]
* Thursday, March 15, 2012 - [[Performance/Snappy/2012-03-15| Minutes]]
* Thursday, March 8, 2012 - [[Performance/Snappy/2012-03-08| Minutes]]
* Thursday, March 1, 2012 - [[Performance/Snappy/2012-03-01| Minutes]]
* Thursday, February 23, 2012 - [[Performance/Snappy/2012-02-23| Minutes]]
* Thursday, February 16, 2012 - [[Performance/Snappy/2012-02-16| Minutes]]
* Thursday, February 9, 2012 - [[Performance/Snappy/2012-02-09| Minutes]]
* Thursday, February 2, 2012 meeting cancelled due to travel
* Thursday, January 26, 2012 - [[Performance/Snappy/2012-01-26| Minutes]]
* Thursday, January 19, 2012 - [[Performance/Snappy/2012-01-19| Minutes]]
* Thursday, January 12, 2012 - [[Performance/Snappy/2012-01-12| Minutes]]
* Thursday, January 5, 2012 meeting cancelled due to holidays
|}

{| class="wikitable collapsible collapsed" style="width: 100%"
! style="background-color: rgb(221, 221, 221);" | 2011
|-
|
* Thursday, December 29, 2011 - meeting cancelled due to holidays
* Thursday, December 22, 2011 - [[Performance/Snappy/2011-12-22| Minutes]], [https://blog.mozilla.org/tglek/2011/12/22/snappy-dec-22/ Progress report]
* Thursday, December 15, 2011 - [[Performance/Snappy/2011-12-15| Minutes]], [https://blog.mozilla.org/tglek/2011/12/16/snappy-update-for-dec-15/ Progress report]
* Thursday, December 8, 2011 - [[Performance/Snappy/2011-12-08| Minutes]], [https://blog.mozilla.org/tglek/2011/12/13/snappy-summary-for-dec-8/ Progress report]
* Thursday, December 1, 2011 - [[Performance/Snappy/2011-12-01| Minutes]], [https://blog.mozilla.org/tglek/2011/12/01/introducing-project-snappy/ Progress report]
* Friday, November 18, 2011 - [[Performance/Snappy/2011-11-18| Minutes]]
|}

== Bug Tracking ==
Bugs tracked by the Snappy project are prioritized by adding one of [Snappy:P1], [Snappy:P2], or [Snappy:P3] to the whiteboard. Use whiteboard [Snappy] tag to nominate bugs.

* [[Performance/Snappy/Dashboard | Snappy dashboard]] - shows recent Snappy changes

* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=Snappy%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized Snappy bugs]. These are triaged as developer free up.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P1 bugs]. These are discussed regularly in meetings.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p1]",
"x_axis_field": "status"
}
</bugzilla>
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P2 bugs]. These are discussed occasionally in meetings. P2 is the default priority.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p2]",
"x_axis_field": "status"
}
</bugzilla>
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=Snappy%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id Snappy:P3 bugs]. These are discussed rarely in meetings.
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy:p3]",
"x_axis_field": "status"
}
</bugzilla>

Some other interesting bugs/lists.
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp;query_format=advanced;status_whiteboard=telemetry-needed;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;resolution=---;resolution=DUPLICATE;list_id=2435528 bugs that need telemetry probes implemented]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=698500 visible unresponsiveness metabug]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=699820 sync storage main-thread-io metabug]
* [https://bugzilla.mozilla.org/show_bug.cgi?id=698919 cycle collector responsiveness metabug]

== Goals ==
* 50ms responsiveness when typing in a textbox ({{bug|703668}})
* 60fps animations on UI operations, i.e. closing tabs ({{bug|702509}})
* track above via telemetry

more goal thinking [https://etherpad.mozilla.org/desktop-responsiveness responsiveness goals]

My (Asa) proposed core measures:
* Startup on par with IE and Chrome
* 50ms responsiveness for all non-animated primary UI interactions (list TBD)
* "Above the fold" uncached page load on par with IE and Chrome
* Back/Forward navigation as fast as Opera.

== Snappy Addons ==
If you discover an addon that may be causing responsiveness issues
# Capture a profile of the jank with [https://developer.mozilla.org/en/Performance/Profiling_with_the_Built-in_Profiler SPS Profiler].
# File a bug under Tech Evangelism/Add-ons. Describe the issue, steps to reproduce. [https://bugzilla.mozilla.org/enter_bug.cgi?product=Tech%20Evangelism bugzilla link].
# Upload your SPS profile + post link in bug
# Contact addon author. Authors are usually happy to make their addons more performant.
# If the developer isn't responsive we will get AMO team involved to consider delisting the offending addon version.

See {{bug|777397}} for an example of above process.

== Wins ==
<bugzilla type="count" display="bar">
{
"whiteboard": "[snappy",
"resolution": "fixed",
"x_axis_field": "target_milestone"
}
</bugzilla>
;Firefox 16
* [http://bit.ly/LOFsXE All fixed bugs]

;Firefox 15
* Start-up optimization - remove prefetch files (backed out in Firefox 16)
* No more font enumeration
* [http://mzl.la/Iwq9xT All fixed bugs]

;Firefox 14
* IE profile migrator rewrite
* Async favicons
* Slow SQL Telemetry
* [http://bit.ly/GGBpJ5 All fixed bugs]

;Firefox 13
* Tabs on demand
* CC/GC pause reductions
* Start-up optimizations
* Front-end Telemetry
* [http://j.mp/wXnf5x All fixed bugs]

;Firefox 12
* <nowiki>about:jank</nowiki>
* Inline autocomplete
* Slow SQL Telemetry
* [http://j.mp/AhayWF All fixed bugs]

;Firefox 11
* [http://j.mp/ypxFzj All fixed bugs]

==Snappy TODO==
===Firefox===
* {{nbug|712478}}: interactivity heuristics (to reduce jank while playing video, scrolling)
* {{nbug|650968}}: fix personas (startup slowdown, possibly other jank)
* {{nbug|729330}}: fix addon manager (startup slowdown)
* {{nbug|699820}}: fix remaining main thread sqlite (significant source of jank)
* {{nbug|743069}}: fix our tab bar responsiveness (switching tabs, tab animations, etc. We lag other browsers in this area)

===Platform===
* {{nbug|627635}} Make localstorage async (Don't allow web pages to jank us)
* {{nbug|692255}} get rid of windows prefetch (so we can enable library reordering for a big startup win)
* {{nbug|722243}} cancellable sql queries (Cancel background queries to improve interactive performance)
* {{nbug|692557}} Fix GFX acceleration lag (Currently GFX acceleration can result in a severe slowdown over non-accelerated case)
* {{nbug|705594}} Font enumeration lag (This can add minutes of lag on some windows/mac machines)
* OFTC on desktop (Needed for smooth animations)
* Necko improvements (cache, connection pools, etc)

== Current Infrastructure ==

* Event Tracing: http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/EventTracer.cpp#38
** Fires events at main thread event loop from background thread, measures response time
** Produces output of the form MOZ_EVENT_TRACE sample <timestamp> <duration>
** Being rolled out as part of Talos Tp5 {{bug|631571}}, will produce a responsiveness metric for "responsiveness while running Tp5"
* Peptest: {{bug|674606}}
** Responsiveness regression test harness
** Allows devs to write tests that perform various actions and tests responsiveness using EventTracer
** [https://wiki.mozilla.org/Auto-tools/Projects/peptest Project page] contains everything you need to know
** Run 'make peptest' from your object directory to run the tests (see [https://wiki.mozilla.org/Auto-tools/Projects/peptest#Running_Tests running tests] for command line instructions)

* [http://people.mozilla.org/~xstevens/telemetry/telemetry-histogram-aggregates-20111103-20111116.txt Telemetry dump for ff11 from November 17]

* [http://graphs-new.mozilla.org/graph.html#tests=[[130,1,1],[130,1,12],[130,1,13],[130,1,21],[130,1,22]]&sel=1322091900253,1322167344714&displayrange=7&datatype=running Talos responsiveness tests] - for details see {{bug|631571}}

== Additional Infrastructure ==

* Have metrics do continuous trend analysis to see if our telemetry stats are getting better over time.

== arewesnappyyet.com ==
* This should use our telemetry data to tell us whether we are meeting our expectations and how far off we are
* {{bug|703669}} tracks the implementation of this site

== Testing ==
Placeholder section for QA and testing.

; [https://wiki.mozilla.org/Performance/Snappy/Testing:Sprint_1#Phase_1:_UI_Interactions Sprint 1 Phase 1]

; Takeaways from [https://wiki.mozilla.org/Performance/Snappy/2011-12-01 2011-12-01 Meeting]
* Goal of Project:
** Big hangs (~30s) are a problem, but small hangs (~5s) may be a bigger win
** Identify where we hang, why we hang, and plug those holes
* Things on Devs Radar:
** Meta Bugs: visible responsiveness ({{bug|698500}}), sync storage ({{bug|699820}}), cycle collector ({{bug|698919}})
** Animation hangs are most visible
** Windows, Mac, Linux are all unique -- Windows is P1 for the project
** [http://benoitgirard.wordpress.com/2011/11/08/firefox-built-in-profiler/ Firefox Built-in Profiler]: late Q1'2012
** Chromhang Bugs: {{bug|705761}}, {{bug|705287}}, {{bug|705258}}, {{bug|705594}}, {{bug|704933}}, {{bug|699051}}
** Small Jank: using ''refresh driver'' to measure refresh-rate performance (ie. throbber, progress bar, etc -- Asa has more ideas)
** Measuring Snappiness: [http://j.mp/rPGyFb Telemetry use cases]
** Metrics/Automation: for trend-analysis, finding regressions with Telemetry
* Where does QA fit in?
** P1: [https://etherpad.mozilla.org/desktop-responsiveness Desktop responsiveness] competitive testing
** many of these problems need stack traces (something Telemetry cannot do)
** crowd-sourcing use cases where we are less Snappy
** A-B testing with other browsers (versions of Firefox, competitor browsers)
** Ownership of Snappy P1s
** Caching issues should have patches landed in the next week or so