Add-ons/QA/Testplan/Add-ons Post Reviews Process: Difference between revisions

Add-ons/QA/Testplan/Add-ons Post Reviews Process (view source)

Revision as of 12:13, 31 August 2017

332 bytes added , 31 August 2017

→‎what's in scope?

ValentinaP

302

edits

@@ Line 49: / Line 49: @@
   - Display recent user ratings (3 stars of fewer) and abuse reports (for the add-on or the developers, if there are any reports), with links to the full lists, below the add-on metadata and right above “More about this add-on”.
   - The "Confirm Approval" resolution should be available, and shouldn’t display the form for comments and canned responses. Instead, it should only show the Save button. Confirming doesn’t send any information to the developer or change its status. It only records it so the last manually-approved version is used to calculate the code changes compared to the latest version.
-  - The Request super-review action shouldn’t send an email to the developer.
+  - Reject Multiple Versions - should allow the reviewer to select a range of versions to reject (disable) with a single review message.
-  - The Reject action should include a way to select which versions of the add-on are affected by the rejection. All of those versions should be disabled once the review is saved, and should be listed in the email sent to the developer.
+ - Reviewer reply - should work the same as with regular reviews. (combined with the possibility of a more info request checkbox option).
+  - Requesting super-review - should increase weight if the add-on wasn’t flagged for super-review before.
+ - Adding a comment - should work the same as with regular reviews.
 '''4. Post-review prioritization'''
 *The post-review list will be sorted according to a weighted sum of the following risk factors:
   - The add-on has the admin review flag.
-  - Flags raised during static analysis.
+  - Flags raised by static validation after webextension submission:
+  - eval(), document.write(), setInterval/setTimeout (with a string, not a function), document.write, innerHTML, or a custom CSP;
   - Size of code changed since last manual approval.
   - User feedback obtained from abuse reports (for the add-on and the developers).
@@ Line 68: / Line 71: @@
 '''5. Submission process updates'''
 *The submission flow will have the following changes:
- - Replace the Developer Agreement step with a step linking to the 3 main documents: Developer Agreement, Review Policy, and Review Rules. Each document will have a checkbox next to it indicating the developer has read them and agreed to them.
+- New submissions should show the new Developer Agreement / Policy / Rules acceptance checkboxes, with links to MDN (the Rules doc might be 404).
- - Ensure all developers see these documents on their next submission, even for an update.
+- The last submission step should indicate the add-on will be available soon and not refer to waiting for review.
- - For WebExtensions, a subset of flags raised by static validation will be shown after it completes. They are specified on this spreadsheet, in the column “Flag with developer during submission”.
+- After submission, the uploaded version should be publicly available on AMO within 15 minutes (probably less time than that).
- - For WebExtensions, the text in the last step should be adjusted to reflect the new process.
+- Check that the add-on status is appropriately updated in the Developer Hub.
+- Check that the add-on appears in the auto-approval list (requires the tester to have the Addons:PostReview permission).
+- check the add-ons and weights to verify they are being calculated correctly based on the spec.
 '''6. Remove auto-approval restrictions'''
-*All WebExtension submissions will be post-reviewed after this point
+* All WebExtension submissions will be post-reviewed after this point
 ===what's out of scope?===