TestEngineering/Performance/NimbleDroid: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(8 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= At a glance =
{{delete|Project is no longer active|date=2020-09-17}}
[https://nimbledroid.com/ NimbleDroid] provides functional performance testing for Android and iOS via a service. We currently use NimbleDroid exclusively for testing the performance of our Android applications on real devices.


{| class="wikitable"
As of around May 2020, we no longer use Nimbledroid: https://github.com/mozilla-mobile/fenix/issues/11050
|-
! Application !! Upload !! [[#Cold startup]] !! [[#Page load]]
|-
| Fennec || {{todo|}} || ✅ || {{todo|}}
|-
| GeckoView Example || {{todo|}} || ✅ || {{todo|}}
|-
| Reference Browser || {{todo|}} || ✅ || {{todo|}}
|-
| Fenix || {{todo|}} || ✅ || {{todo|}}
|-
| Chrome || {{todo|}} || ✅ || {{todo|}}
|}


= Test scenarios =
We've switched to an in-house system instead (FNPRMS, to be replaced by mach perftest) because:
 
* it's difficult to debug performance when working with a black box
== Cold startup ==
* Nimbledroid tests run with <pre>debuggable=true</pre> and that provides unreliable performance characteristics
The cold startup scenario is profiled for every app. See the [http://docs.nimbledroid.com/#cold-startup NimbleDroid documentation] for details on what and how they measure for cold startup.
 
== Page load ==
Custom scenarios are used to test page load for the following sites:
# https://abcnews.go.com/
# https://m.yelp.de/
# {{todo|}}
 
= Uploading apps =
 
== Automated ==
{{todo|}}
 
== Manual ==
{{todo|}}
 
= Results =
The test results can be seen in [[../Results#Firefox_Health|Firefox Health]] and [[../Results#Nimbledroid|NimbleDroid]]'s dashboard.
 
= Alerting =
{{todo|}}

Latest revision as of 10:50, 18 August 2021

Do not edit this page THIS PAGE IS PROPOSED FOR DELETION 2020-09-17
Project is no longer active

As of around May 2020, we no longer use Nimbledroid: https://github.com/mozilla-mobile/fenix/issues/11050

We've switched to an in-house system instead (FNPRMS, to be replaced by mach perftest) because:

  • it's difficult to debug performance when working with a black box
  • Nimbledroid tests run with 
    debuggable=true
    and that provides unreliable performance characteristics
Retrieved from "https://wiki.mozilla.org/index.php?title=TestEngineering/Performance/NimbleDroid&oldid=1237385"