Review Decision Guidelines

This page provides guidance to support reviewers in making consistent review decisions based on our Review Policies. In addition, it explains how to best communicate with the developer when conducting add-on reviews. It is a supplement to our Reviewer Guide.

Completing the Review

Add-ons must be reviewed completely, noting issues in the review comments field as you find them. We want to avoid sending multiple partial reviews to developers and give them an opportunity to address all policy violations at once in their next submission.

In the following sections we will explain how canned responses should be used for the issues you find and guide you through the review actions. Finally, you are guided on how to respond if you find issues that cover multiple review actions.

Use of Canned Responses

To ensure developers receive consistent, actionable responses from the review team, AMO makes use of canned responses that cover many of our policy issues.

If no canned response is available for the issue you have found, you may write a custom response specific to the situation. Keep in mind that when writing a response to the developer, all responses must be in line with our policies. This is especially true when using custom responses. If you are unsure your response conforms with the policy, an admin reviewer will be able to assist you.

In addition, we’d love to hear from you believe the response is useful to add to the official list of canned responses. We would much prefer that reviewers use the canned responses to ensure consistent replies, therefore adding new canned responses could be very useful.

In some cases the developer responds to your review, either via email or by uploading a new version, making clear that they did not fully understand how to act upon the canned response. In such cases, sending the same canned response again will not help resolve the situation. Here it makes sense to explain the policy issue in a different, possibly more verbose way, using your own words.

Example

While the review comments field is free-form, we have a de-facto standard approach that lists the review issues in numbered lists each with a headline. Note that your text is inserted into an email that handles all the formalities, you should not need to begin your review with “Dear Developer” or sign with your name.

Commonly, we use a numbered list item for each review issue, along with headings to separate sections as needed. Here is an example review text to give you an idea:

This add-on didn't pass review because of the following problems:

1) We don't allow add-ons to use remote scripts because they can create serious security vulnerabilities. We also need to review all add-on code, and this makes it much more difficult. Please insert those scripts locally from your add-on code.
- pages/inject.html line 8

In addition, the following information is required to complete the review:

1) Please provide us with detailed information on how to test this add-on. If authentication to a website is necessary, give us a test username and password to facilitate our work. This can be provided in the Whiteboard field, which can be found in the Edit Listing page under the Technical Details section. This information is only available to reviewers.

Reviewer Replies

With a reviewer reply, you can convey information to the developer. You can use this action to answer questions the developer may have about your review. All communication between developers and reviewers is captured on the review history page.

If you need the developer to take action, please make use of a delayed rejection instead. Doing so makes sure the correct email template is used and AMO knows to remind us when the developer has not responded to the request.

If you are unsure how to answer a response from a developer, or the developer is disputing your review, please get in touch with admins to make sure the developer receives a reply. Involving a second reviewer to confirm your review can also be helpful if the developer does not want to accept your reasoning.

Delayed Rejection

If the add-on has minor policy issues that don’t require an immediate rejection, the delayed rejection action should be used. AMO will then reach out to the developer, asking them to resolve the issue within the requested time frame. If the developer fails to comply, the marked versions will be rejected.

To use a delayed rejection, select “Reject Multiple Versions”, and select the option to delay the rejection. All versions you have selected will be rejected after the time passes. You will need to determine when the developer started to use the code in question, so you can determine which versions to reject. Generally it is sufficient to check back until the last reviewed version, for example using a bisection approach. If the issue also exists in that last reviewed version, you will need to track back further.

If the developer responds, you will receive email with the developer’s response. At this time you should return to the review page to continue reviewing using the information you received from the developer. Note that the developer may follow up with further questions about the information you are requesting. In all cases, please answer developers in a timely manner (within 2 days). If you are answering questions that require no further action or response, please use the “reviewer reply” action.

If you cannot complete the review after the developer has provided information, please at least check if the developer has provided the information you have requested and leave a comment on the version so that other reviewers can continue your work.

Similarly, if you come across an add-on where the developer has provided information but the original reviewer has not followed up within that time, you are welcome to complete the review. The original reviewer may have prior knowledge that would speed up the review, but at the same time we do not want to keep developers waiting.

Immediate Rejection

Rejecting an add-on’s versions can mean anything from showing an outdated version of the add-on to completely hiding the listing from addons.mozilla.org (when rejecting all public versions). Especially in the post-review model where add-ons are reviewed after initial approval, a rejection can lead to frustration from the developer. At the same time, we need to ensure that there are no policy violations that threaten user security or privacy.

Our general policy is to only reject immediately when necessary. Rejection is necessary when an add-on has security or privacy issues, doesn't meet our content policies, or fits one of the examples described later on.

When rejecting the add-on, you will need to determine all affected versions using the same approach as for a delayed rejection. You can then select the option to reject immediately.

In case of a rejection, developers may have questions on how to best resolve the policy issues, or if they have trouble understanding the message. Similar to the delayed rejection, please answer in a timely manner using the reviewer reply feature.

Multiple Categories of Issues

If you find multiple issues within the add-on where there are both delayed and immediate rejections, you may reject the add-on immediately. Please make sure to clearly separate information needed to complete the review from rejection reasons. An example on how this can be done is shown above.

Super-Review

Using the “Request Super Review” action will put the add-on into the admin queue. There are few select cases where this would occur, please see the examples below for further details. If you have a concern that needs immediate admin attention, please get in touch with the admins via the channel described in the Escalation section, as there are no specific notifications to admins when you request super review.

Escalation

There are certain cases that are severe enough that the reviewer actions on addons.mozilla.org are not sufficient and you must inform the admin team. This is especially true for reporting child pornography, add-ons containing abusive functionality or being of malicious nature that must be blocked in accordance with our blocking policy.

In such cases, you should contact the admin team via email at amo-admins [at] mozilla [dot] com. You can find more examples on when to escalate later on this page.

Examples

The following sections show a few common examples on how to respond to certain policy violations. Please note however these are merely examples intended to convey the intent we have with the policies. It should not be considered a complete list of review decisions.

No Surprises

Content Policy

The content review policies are detailed in separate guidelines. Here are a few select examples for the content policy:

Submission Guidelines

Development Practices

Data Disclosure, Collection and Management

This section has a few items related to the privacy policy. We do not check the privacy policy for correctness. We do however make sure the privacy policy is more than just a link and generally about the add-on.

Additional Privacy Protocols

Monetization

Monetization follows the same data disclosure policies as for other data and includes a few extra provisions to set user expectations. We define monetization as a feature of the add-on that results in a potential monetary benefit for the developer.

Security, Compliance & Blocking

Previous: Reviewing Next: Moderation