302
edits
Add-ons/QA/Testplan/Add-ons Post Reviews Process: Difference between revisions
Add-ons/QA/Testplan/Add-ons Post Reviews Process (view source)
Revision as of 12:23, 31 August 2017
, 31 August 2017→what's in scope?
ValentinaP (talk | contribs) |
ValentinaP (talk | contribs) |
||
| Line 38: | Line 38: | ||
== Scope == | == Scope == | ||
===what's in scope?=== | ===what's in scope?=== | ||
'''1. Deploy Auto-approval''' | '''1. Deploy Auto-approval''' | ||
WebExtensions will continue to be submitted through the regular flow, and a command will be run regularly (~every hour) to evaluate and auto-approve some of them, based on criteria defined in the linked [https://docs.google.com/document/d/1VxvvzQQvfDOVCylSVlpfeXv0tDYRLZfIMvWOoraANGU/edit#heading=h.diha6b7e1on6 PRD]. | WebExtensions will continue to be submitted through the regular flow, and a command will be run regularly (~every hour) to evaluate and auto-approve some of them, based on criteria defined in the linked [https://docs.google.com/document/d/1VxvvzQQvfDOVCylSVlpfeXv0tDYRLZfIMvWOoraANGU/edit#heading=h.diha6b7e1on6 PRD]. | ||
'''2. Implement post-review list for auto-approved add-ons''' | '''2. Implement post-review list for auto-approved add-ons''' | ||
Add-ons that are auto-approved will appear in the post-review queue from [https://addons-dev.allizom.org/en-US/editors/queue/auto_approved Reviewer Tools]. List will contain: add-on name and version number (linking to the corresponding review page), last review (time since last manual review - in days), Flags and Weight (sorted after weight) | Add-ons that are auto-approved will appear in the post-review queue from [https://addons-dev.allizom.org/en-US/editors/queue/auto_approved Reviewer Tools]. List will contain: add-on name and version number (linking to the corresponding review page), last review (time since last manual review - in days), Flags and Weight (sorted after weight) | ||
'''3. Changes to Reviewer Tools''' | '''3. Changes to Reviewer Tools''' | ||
If the last review for an add-on was done manually or the user looking at the page doesn’t have the Addons:PostReview permission, show the current reviewer page. | If the last review for an add-on was done manually or the user looking at the page doesn’t have the Addons:PostReview permission, show the current reviewer page. | ||
If the last review for an add-on was automatic and the user looking at the page has the Addons:PostReview permission, show the reviewer page with the following changes: | If the last review for an add-on was automatic and the user looking at the page has the Addons:PostReview permission, show the reviewer page with the following changes: | ||
# Display recent user ratings (3 stars of fewer) and abuse reports (for the add-on or the developers, if there are any reports), with links to the full lists, below the add-on metadata and right above “More about this add-on”. | # Display recent user ratings (3 stars of fewer) and abuse reports (for the add-on or the developers, if there are any reports), with links to the full lists, below the add-on metadata and right above “More about this add-on”. | ||
# The "Confirm Approval" resolution should be available, and shouldn’t display the form for comments and canned responses. Instead, it should only show the Save button. Confirming doesn’t send any information to the developer or change its status. It only records it so the last manually-approved version is used to calculate the code changes compared to the latest version. | # The "Confirm Approval" resolution should be available, and shouldn’t display the form for comments and canned responses. Instead, it should only show the Save button. Confirming doesn’t send any information to the developer or change its status. It only records it so the last manually-approved version is used to calculate the code changes compared to the latest version. | ||
# Reject Multiple Versions - should allow the reviewer to select a range of versions to reject (disable) with a single review message. | # Reject Multiple Versions - should allow the reviewer to select a range of versions to reject (disable) with a single review message. | ||
# Reviewer reply - should work the same as with regular reviews. (combined with the possibility of a more info request checkbox option). | # Reviewer reply - should work the same as with regular reviews. (combined with the possibility of a more info request checkbox option). | ||
# Requesting super-review - should increase weight if the add-on wasn’t flagged for super-review before. | # Requesting super-review - should increase weight if the add-on wasn’t flagged for super-review before. | ||
# Adding a comment - should work the same as with regular reviews. | # Adding a comment - should work the same as with regular reviews. | ||
'''4. Post-review prioritization''' | '''4. Post-review prioritization''' | ||
The post-review list will be sorted according to a weighted sum of the following risk factors: | The post-review list will be sorted according to a weighted sum of the following risk factors: | ||
# The add-on has the admin review flag. | # The add-on has the admin review flag. | ||
# Flags raised by static validation after webextension submission: | # Flags raised by static validation after webextension submission: | ||
| Line 69: | Line 69: | ||
The reputation is an integer ranging between 0 and 3, that is set per-add-on, defaulting to 0. | The reputation is an integer ranging between 0 and 3, that is set per-add-on, defaulting to 0. | ||
'''5. Submission process updates''' | '''5. Submission process updates''' | ||
The submission flow will have the following changes: | The submission flow will have the following changes: | ||
# New submissions should show the new Developer Agreement / Policy / Rules acceptance checkboxes, with links to MDN (the Rules doc might be 404). | # New submissions should show the new Developer Agreement / Policy / Rules acceptance checkboxes, with links to MDN (the Rules doc might be 404). | ||
# The last submission step should indicate the add-on will be available soon and not refer to waiting for review. | # The last submission step should indicate the add-on will be available soon and not refer to waiting for review. | ||
# After submission, the uploaded version should be publicly available on AMO within 15 minutes (probably less time than that). | # After submission, the uploaded version should be publicly available on AMO within 15 minutes (probably less time than that). | ||
# Check that the add-on status is appropriately updated in the Developer Hub. | # Check that the add-on status is appropriately updated in the Developer Hub. | ||
# Check that the add-on appears in the auto-approval list (requires the tester to have the Addons:PostReview permission). | # Check that the add-on appears in the auto-approval list (requires the tester to have the Addons:PostReview permission). | ||
# Check the add-ons and weights to verify they are being calculated correctly based on the spec. | # Check the add-ons and weights to verify they are being calculated correctly based on the spec. | ||
'''6. Remove auto-approval restrictions''' | '''6. Remove auto-approval restrictions''' | ||
All WebExtension submissions will be post-reviewed after this point | All WebExtension submissions will be post-reviewed after this point | ||