canmove, Confirmed users
1,220
edits
B2G/QA/API Permissions Test Plan: Difference between revisions
B2G/QA/API Permissions Test Plan (view source)
Revision as of 13:30, 26 November 2012
, 26 November 2012→Suite 2: Verify API behavior
(Created page with "= Boot2Gecko API Permissions Test Plan = == Summary == {| class="fullwidth-table" |- | style="width:20%" | '''Leads''' | [mailto:gmealer@mozilla.com Geo Mealer] (irc: geo)<br...") |
Ptheriault (talk | contribs) |
||
| (7 intermediate revisions by one other user not shown) | |||
| Line 5: | Line 5: | ||
|- | |- | ||
| style="width:20%" | '''Leads''' | | style="width:20%" | '''Leads''' | ||
| [mailto: | | [mailto:dchan@mozilla.com David Chan] (irc: dchan)<br /> | ||
[mailto: | [mailto:gmealer@mozilla.com Geo Mealer] (irc: geo)<br /> | ||
[mailto:ptheriault@mozilla.com Paul Theriault] (irc: ptheriault) | [mailto:ptheriault@mozilla.com Paul Theriault] (irc: ptheriault) | ||
|- | |- | ||
| Line 24: | Line 24: | ||
This test plan takes a more isolated approach to testing permissions interaction, so that AUT instability doesn't block the entire test approach. As such, it's split into three major sections: what permissions look like by default, whether a (non-default) permission setting is properly honored by the API, and whether installed apps properly set permissions in the manager. | This test plan takes a more isolated approach to testing permissions interaction, so that AUT instability doesn't block the entire test approach. As such, it's split into three major sections: what permissions look like by default, whether a (non-default) permission setting is properly honored by the API, and whether installed apps properly set permissions in the manager. | ||
These test suites will likely be implemented in | These test suites will likely be implemented in mochitest, with a possible dependency on mochitest-chrome for the third suite to allow completion of install flow. | ||
== Suites == | == Suites == | ||
| Line 33: | Line 33: | ||
'''Tracking Bug''' | '''Tracking Bug''' | ||
* | * {{bug|811141}} | ||
'''Methodology''' | '''Methodology''' | ||
| Line 50: | Line 50: | ||
'''Tracking Bug''' | '''Tracking Bug''' | ||
* | * {{bug|815105}} | ||
'''Methodology''' | '''Methodology''' | ||
| Line 62: | Line 62: | ||
* Permissions Manager | * Permissions Manager | ||
===Suite 3: | ===Suite 3: Verify permissions database population=== | ||
Given an app context and manifest, does the permissions database populate correctly? | Given an app context and manifest, does the permissions database populate correctly? | ||
'''Tracking Bug''' | '''Tracking Bug''' | ||
* | * {{bug|811141}} | ||
'''Methodology''' | '''Methodology''' | ||
| Line 78: | Line 78: | ||
* Installation flow | * Installation flow | ||
* Permissions Manager | * Permissions Manager | ||
== Design Notes == | |||
In combination, these suites give similar assurance as round trip, but have distinct advantages on a dependency front. The biggest dependency becomes the permissions manager. Even with APIs broken we get verification of default permissions and population, and even with install flow broken we get verification of APIs. | |||
The biggest downside of this approach is that APIs that check the app context directly (as opposed to honoring the permissions manager) will cause problems. If the API blocks the content context unconditionally, it can't be tested via Suite 2. If it passes some other context unconditionally, we wouldn't catch it here. | |||
Augmenting Suite 2 by testing from each of the different contexts should catch each of those scenarios, but reintroduces a dependency on install flow in that suite. Once the code base has stabilized, this would be a worthy expansion to the tests. | |||