This document describes the process to discover, analyze and reach out to Web sites that are not working consistently between Firefox for Android and other mobile browsers.
We want to insure that Firefox for Android users have as good an experience (or better) as when using a competing browser. For Web sites that need to be fixed, we want to push standard solutions that will allow any standards-compliant mobile web browser to display the web site correctly.
We want to do it with responsibility: we want to open the Web for all actors by pushing the use of standards. Except when this is not possible, we shouldn't recommend modifications that would fix the Web site only for Firefox!
The process is divided in three phases:
- Find problematic web sites: identify which web sites are not working well in Firefox for Android
- Triage / Analyze the problem: why the site it not working well, and prioritize the severity of the issues the site has in the browser
- Outreach: reach out to the site to notify them of the problems, provide recommendations on what they can do to fix the issues, and give them information about why it would be good for their site & the web to fix them.
Phase 1: Detect web sites that have issues in Firefox for Android
This phase consists of testing Web sites and determining if the experience in the Firefox for Android browser is correct or not.
There is a testing guide available that details the steps to take when testing web sites on mobile devices: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_for_Android/Compatibility_Testing
For many regions, you can start with the sites listed on the "Are We Compatible Yet" site: http://people.mozilla.org/~lmandel/arewecompatibleyet and test their compatibility in your locale (if they are localized for your country/region/language).
How to Participate in this Phase:
- Test for 1/4 hour (or longer). Take a short break to test & find bugs. Be sure to report what you find in Bugzilla.
- Regional competition: Show progress in your locale by comparing to others (25% of the Top-100 pl Web sites tested were compatible vs 12% of the sk ones).
- Organize test days in your city/town/region - get people together to test websites and report issues.
Who can help: anybody with an Android phone -> crowdsourcing
Result: bugs documenting non-working web sites globally.
- Is there a way to report issues without a bugzilla account?
- Can we have Mozilla Reps in different countries create a Top-100 for their countries (beyond what is in the Are We Compatible Yet list)
Phase 2 : Triage / Analyze the Problems with each site
This is the most complex phase as the issues and solutions need to be carefully analyzed and reported, or the information given to the problematic Web site will be incorrect or incomplete. It is also important to realize that not everybody who receives notification about issues with their web site will have the technical background to make the recommended changes. There may be questions about how much work it will take to fix the site. etc. These should be covered in the email as well as the reference sites we provide to the site owner/developer.
There are two sub-phases:
- reproducing the problem -> anybody can help do this (crowdsourcing).
- Look up bugs, go to the site, reproduce and verify the issues. Make a note in the bug that it has been tested and verified.
- Defining and categorizing the problems -> This is the critical element and the resource bottleneck
- UA: different web sites Android/iPhone/Fx
- layout: something doesn't look ok, what.
- Behavioral problem: an action doesn't work.
- Library problem
How to participate in this phase:
- Pick a web site bug to analyze (or analyze your own bugs from Phase 1): add [Analysis in progress] into the whiteboard and put themselves under Assignee.
- Document the causes of the problems in the bug. When you have solved the issues, change the whiteboard to [Analysis done]
- The mobile team runs weekly metrics on new [Analysis in progress], new [Analysis done] and list of [Analysis in progress] older than two weeks to see if the assignee needs some help solving the issue.
Tools are needed to make this process more efficient
- Call for help: assignee should be able to ask more skilled people to help (devs) -> process to establish (?)
- Automatic tools to help (detection of Webkit-only properties, detection of different site for different UA, even if it may be imperfect)
- Documentation of the problems and how to check them.
Phase 3 : Outreach
Once we know what's happening on the Web site (that is when the analysis is complete, and [Analysis complete] is added in the whiteboard, assignee = nobody, we can work on reaching the site owners.
It is a generally a long process: most Web sites don't respond, and in general the larger they are the less response we get. They will sometimes silently fix the problem, so even without a response we should periodically re-test them.
The critical point here is to reach the right person inside the organization who can fix the problems. There are a couple ways to find the correct contact email.
- Is there a contact address, or contact form on the web page, use it.
- Search for a web development or web marketing department contact in the organization (try LinkedIn, conference speakers, etc.)
- If you are unable to find any contact information, indicate there is no email address available in the bug. This will trigger an internal call for help. Someone may have contacts with them and can get you the information.
How to participate in this phase:
Once you have the contact information, create an email using one of the letter templates (link?) provided. Personalize the letter and add as much detail as possible from the bug on the nature of the problem and the recommended steps for solving the issues. Be detailed. The more good information you provide the more likely they will act on it.
- Once the letter is sent, add [Letter sent] in the whiteboard and a bugzilla entry with the date and email address (if public). This will allow us to track the process.
- Once they respond and we have a working address, we should add it to a private list (in case further problem appear). But if the response was private, do not share the address publicly! Indicate in bugzilla that a (non automated) response have been received and what kind of response. Add [Contact established] to the whiteboard.
- If they don't respond after one week, resend the message, they may have skipped or lost it.
- If no success after the second try, try sending from a different email address in case yours is being filtered. You cn ask for someone else to send in the bug, just remove yourself as the assignee and note there needs to be another attempt at contact.
- If this doesn't work, we should call for help (Moz Reps channel, escalation), or just wait for a month or two and try again.
Special case for library problem: these are critical cases as they are likely affecting numerous Web sites, and we should use our devengage team to reach them. Also some are open source, so we should see if we can propose a patch to fix the problem.
- Finally, when the web site is fixed: update the bug to [Resolved/Fixed or Resolved/Worksforme] and clear entries in the whiteboard so they do not continue to show up in the metrics reports.