User:ChrisHofmann/InternationTestingStrategy

Draft -- please contribute if you have ideas or input on this topic

This document outlines the people, tools, planning, process and general test coverage areas needed to successfully deliver high quality Firefox releases in many locales around the world. It is written to help gather and share best practices among localization teams and to raise visibility of key work and increase participation in testing of localized builds.

People

Contact Info for Testing Leaders and Contributors

All people participating in testing for any locale should be identified as contacts and each localization team should try to establish a testing leader to help coordinate work among testing contributors. The current best place to identify locale testing contributors are the pages under https://wiki.mozilla.org/L10n:Teams

Local Forums for Testing Discussion and Feedback

Each localization testing team should also attempt to set up feedback forums to help coordinate work and gather general testing feedback. A policy should also be set up to make sure that feedback gets translated into bugs and a bugzilla is set up to track issues. (e.g. https://bugzilla.mozilla.org or a locale specific instance of bugzilla.

We should encourage and promote setting up these forums and discussion in the native language of the locale to encourage the widest participation in the testing effort. Many teams already have these forums set up. Mozilla will attempt to help run hosting services for this kind of activity if needed though initiatives like http://contribute.mozilla.org/Mozilla_Community

Localization Test Team Community Building

Project leaders and Localization Testing Team Leaders should actively recruit testing contributors from Universities, Conferences, and other Open Source Projects to help build testing teams within any Locale. The goal is to get as much participation as possible during the early stages of a release, and to mobilize a wide variety of testing coverage.

International Test Coverage Areas

Internationalization Test Coverage

International issues span multiple localizations, like, European Latin Script localizations, Devanagari script, rtl, etc. A short discussion around the testing issues and needs of each area follows.

Localization Test Coverage Areas

Translation and User Experience Testing

This section covers needs and issues around topics like "Is the wording of this dialog ambiguous? Do I know what Firefox wants?" Problems in this area can be in the lack of clearity of the orginal english text, or problems in bringing the right context to the translated version.

Tools like spelling and grammar checkers can help but beyond that this area will be more subjective. The key is to have systems set up to gather feedback and data; and then be able to use that data to identify any hot spots in the product that if changed will improve understanding and use of the product by many users.

Locale Customizations

The rational for Locale Customizations is to help provide the best set search engines, RSS feeds, and other services that are most useful within both the language and region where the product will be used.

The goal is to provide the best possible user experience, and work toward that goal is accomplished though suggestion and peer review of the service. Once the service is select, and approved by the provider there are implementation details that need to be followed for the features to work correctly.

The sections below specify strategies for checking each of the customization areas.

Other Issues =

Platform Coverage

This section might cover particular issues around line ending formats of build and configuation files and other problems in the build process.

It should also talk about the need to recruit people with specific platform knowledge to help test on Windows, Mac, and Linux systems for each shipping locale.

International Test Planning

This section covers raising awarness about the importance of PRD's, Specs, Bugs, and other documentation that covers the breadth of changes in a release.

Some examples: https://wiki.mozilla.org/Firefox3/Firefox_Requirements https://wiki.mozilla.org/Firefox3/Product_Requirements_Document https://wiki.mozilla.org/Firefox3.1/Features

With the combination of documents and information listed above a test planner can review the types of changes being made in a release, and combine with the check lists of International Test Coverage Areas to identify testing cases and procedures that will be needed for a release under development. The test planner can begin to draw special attention to key the areas that will testing before final release.

Timing of this test planning activity will vary, but should be started as soon as the first features of a release start to appear in alphas or beta1

International Testing Tools

Automated and Non-automated tools can help to cover things like spell checking of localized content, dialog size errors, search engine, rss feeds, and other customizations for each locale. This section covers an introduction to these tools.