97
edits
Performance/Fenix/Performance reviews: Difference between revisions
Performance/Fenix/Performance reviews (view source)
Revision as of 12:42, 29 March 2023
, 29 March 2023Add information for backfilling performance regressions.
(Add information about how to run CI performance tests on custom APKs.) |
(Add information for backfilling performance regressions.) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 24: | Line 24: | ||
** This will replace the APK used in mozperftest tests (e.g. startup tests). | ** This will replace the APK used in mozperftest tests (e.g. startup tests). | ||
** Use <code>--browsertime-upload-apk</code> if you want to target Browsertime performance tests. | ** Use <code>--browsertime-upload-apk</code> if you want to target Browsertime performance tests. | ||
* '''Commit the changes in-tree with: <code>hg commit -m "Upload APK"</code>''' | * '''Commit the changes in-tree with: <code>hg bookmark my-upload; hg commit -m "Upload APK"</code>''' | ||
** Note that this will move you out of the default branch/bookmark (central, or unified). To return to the original bookmark, run <code>hg update central</code>. | |||
** You can find a great [http://guide%20for%20how%20tohttps://mikeconley.github.io/documents/How_mconley_uses_Mercurial_for_Mozilla_code.html%20use%20Hg%20written%20by%20Mike%20Conley%20%5B:mconley%5D%20her guide for how to use Hg written by Mike Conley [:mconley<nowiki>]</nowiki> here]. | |||
* '''Now you can re-run the performance selector to pick the tests you want to run, and perform the push: <code>./mach try perf --android --show-all</code>''' | * '''Now you can re-run the performance selector to pick the tests you want to run, and perform the push: <code>./mach try perf --android --show-all</code>''' | ||
** Search for <code>'perftest 'start</code> to find the startup tests. | ** Search for <code>'perftest 'start</code> to find the startup tests. | ||
| Line 133: | Line 135: | ||
With the measurement in hand, repeat these steps for your changes and compare the resulting times. Note: it's possible the device was under load when you took the profile so you may wish to take more than one profile if you suspect that is the case. | With the measurement in hand, repeat these steps for your changes and compare the resulting times. Note: it's possible the device was under load when you took the profile so you may wish to take more than one profile if you suspect that is the case. | ||
== Backfilling to determine culprit commit == | |||
We now have alerting enabled on the firefox-android branch/project. If any changes are detected, an alert will be produced after ~6 days (once enough data is produced). With the alert, you'll need to determine the culprit commit, or the commit that caused the regression. | |||
To do so, you can start with backfilling on the firefox-android branch. "Backfilling" is the act of running the regressing test on past pushes that didn't run the test. This lets you fill in all the holes you have in your data (or backfilling them). It's likely that you'll find a different culprit commit than the one you originally found. | |||
If the culprit commit suggests that there's something coming from mozilla-central/geckoview, then you'll need to move to mozilla-central and start searching for the culprit commit there with the same backfilling process. That said, you'll need to use try runs to do this with custom Fenix APKs that are built using the current mozilla-central/geckoview commit. You should be able to find geckoview artifacts in autoland/mozilla-central so you won't need to remake them. See the [[#Benchmark remotely|Benchmark remotely]] section for information about how you can do those try runs. Please reach out in [https://matrix.to/#/#perftest:mozilla.org| #perftest] if you need any help, or if anything is unclear. | |||