Confirm
2,473
edits
Changes
→Tracking Progress Though Effective Metrics
== Tracking Progress Though Effective Metrics ==
We need a tool for tracking progress of the testing, engineering, and evangelism efforts. The tool should make it clear what actions are required to make a site compatible. The tool will capture the following bits of information.
List of the sites we are consistently testing against. (About 1700?)
- it would be useful to be able to categorize the sites (alexa, phonegap, etc.)
For each of those sites:
* which browser where used in the evaluation
* overall evaluation of the site. e.g. site is working, minor visual problems, major display problems, interaction problems, not getting right content
** break site problems down by category: UA sniffing (wrong site served), visual problems (-webkit or general layout issues), performance issues (probably layout), interaction (touch or other functionality)
* links to bugs found in the testing
** let's tag all of the bugs so that we can easily query for them. we can use site-compatibility, top-apps, something else?
* who has the next action on the site (QA, Evangelism, Engineering)
** there may be multiple next action if we need to follow up on UA and fix a layout issue
* do we have a contact at the site?
** who has the contact, or is in contact with the site.
We need to develop some metrics for monitoring progress:
Stats could be broken down to: mobile aware, but not fennec aware vs. not mobile aware at all.
Wishlist:
* automated tool to perform regression testing on a regular schedule (weekly)
* table that displays sites and issues. columns including: site name/URL, UA identified correctly (yes/no), layout correct (yes, no + reason -webkit, performance, other), functional interaction (yes/no), open bugs
** for bonus marks allow filtering of table based on site categories, issue type
==Documentation==
