Confirmed users
1,333
edits
Simona Badau (talk | contribs) |
Simona Badau (talk | contribs) |
||
(41 intermediate revisions by 2 users not shown) | |||
Line 7: | Line 7: | ||
! Date !! Version !! Author !! Description | ! Date !! Version !! Author !! Description | ||
|- | |- | ||
| | | 02/23/2017 || 1.0 || Simona Badau || Created first draft | ||
|- | |- | ||
| | | || || || | ||
|} | |} | ||
Line 25: | Line 25: | ||
== Scope == | == Scope == | ||
This wiki details the testing that will be performed by the project team for the | This wiki details the testing that will be performed by the project team for the Use a separate content process for file:// URLs project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document: | ||
* What will be tested | * What will be tested | ||
* How testing will be performed | * How testing will be performed | ||
Line 32: | Line 32: | ||
• Product Manager: | • Product Manager: | ||
* | * Jim Mathies (:jimm) | ||
• Project Manager: | • Project Manager: | ||
* | * Jim Mathies (:jimm) | ||
• Engineering: | • Engineering: | ||
* | * Jim Mathies (:jimm) | ||
• Developer: | |||
* Bob Owen (:bobowen) | |||
• UX: | • UX: | ||
* | * N/A | ||
• QA Engineering Team - Nightly Testing | • QA Engineering Team - Nightly Testing | ||
Line 51: | Line 54: | ||
== Scope of Testing == | == Scope of Testing == | ||
=== In Scope === | === In Scope === | ||
'''Use a separate content process for file:// URLs''' feature aims to make some changes in the way the content is handled when loading file:// URIs. | |||
That is content loaded from the local file system hat is now loaded in a separate child content process. | |||
The feature is controlled by the boolean preference browser.tabs.remote.separateFileUriProcess. | |||
Another relevant preference is dom.ipc.processCount.file that allows multiple file content processes. | |||
=== Out of Scope === | === Out of Scope === | ||
Testing the other types of content processes that are not related to the file:// URIs such as processes allocated for web sites, extensions etc. | |||
= Requirements for testing = | = Requirements for testing = | ||
== Environments == | == Environments == | ||
Full testing will be performed on the following OSes: | Full testing will be performed on the following OSes: | ||
* | * Windows (Windows 7 and Windows 10) | ||
* | * Ubuntu 16.04 | ||
* | * Mac OS X (Mac OS X 10.11 and Mac OS X 10.12) | ||
Other configuration/environmental setup needed: | Other configuration/environmental setup needed: | ||
* In Firefox configuration settings, '''''browser.tabs.remote.separateFileUriProcess''''' preference needs to be set to '''''true'''''. | * In Firefox configuration settings, '''''browser.tabs.remote.separateFileUriProcess''''' preference needs to be set to '''''true'''''. | ||
* To allow multiple content processes, change the value of the preference '''''dom.ipc.processCount.file'''''. | |||
= Test Strategy = | = Test Strategy = | ||
Line 73: | Line 82: | ||
! ID !! Description / Threat Description !! Covered by Test Objective !! Magnitude !! Probability !! Priority !! Impact Score | ! ID !! Description / Threat Description !! Covered by Test Objective !! Magnitude !! Probability !! Priority !! Impact Score | ||
|- | |- | ||
| RAC-1 || || || || || || | | RAC-1 || A local content process is not opened when a local file is opened || TO-1, TO-2, TO-3, TO-4, TO-5, TO-6|| '''3-High''' || ''2-Possible'' || ''2-Medium'' || 12 | ||
|- | |||
| RAC-2 || A local content process is not closed when it's correponding local file is closed || TO-1, TO-2, TO-3, TO-4, TO-5, TO-6 || '''3-High''' || ''2-Possible'' || ''2-Medium'' || 12 | |||
|- | |- | ||
| RAC-2 || || || || || || | | RAC-3 || The content process is not changed when navigating from a local file to a web site and viceversa || TO-3, TO-4 || '''3-High''' || ''2-Possible'' || ''2-Medium'' || 12 | ||
|- | |||
| Rac-4 || The interaction between the different types of content processes || TO-3, TO-4 || ''2-Moderate'' || ''2-Possible'' || 1 - Low || 4 | |||
|- | |- | ||
|} | |} | ||
Line 99: | Line 111: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! Ref !! Function !! Test Objective !! Test Type !! Risk Assessment and Coverage(RAC) !! Owners | ! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! Risk Assessment and Coverage(RAC) !! Owners | ||
|- | |||
| TO-1 || A local file is opened || The content of a local file is loaded in a separate child content process (local content process) || The local file is opened and it's content is loaded in a local content process allocated for local files || Manual || RAC-1 || Eng Team | |||
|- | |||
| TO-2 || Multiple local files are opened || The content of multiple local files are opened in one or more local content processes (depending on the value of pref dom.ipc.processCount.file) || Local files are opened in the content processes allocated for the local files || Manual || RAC-1 || Eng Team | |||
|- | |||
| TO-3 || Local files and websites can be opened in the same tab || Local files and websites are opened in the same tab and the type of the content process differs (local content processes and websites content processes) || Local files and websites are opened in the same tab and the type of the content process differs (local content processes and websites content processes) || Manual || RAC-1, RAC-2, RAC-3 || Eng Team | |||
|- | |- | ||
| TO- | | TO-4 || Local files and websites can be opened in different tabs || Local files and websites are opened in different tabs || Local files and websites are opened in different tab and the type of the content process differs (local content processes and websites content processes)|| Manual || RAC-1, RAC-2, RAC-3 || Eng Team | ||
|- | |- | ||
| TO- | | TO-5 || Accessibility || Local files and their correspondent local content processes are not affected from the accessibility point of view || Testing for accessibility includes: <br /> | ||
- Screen Reader validation<br /> | |||
- Keyboard-only | |||
|| Manual || RAC-1, RAC-2 || Eng Team | |||
|- | |||
|TO-6 || Local file is closed || Local files are closed along with their corresponding local content processes || The local content processes are closed when the local files are closed || Manual || RAC-2 || Eng Team | |||
|- | |- | ||
|} | |} | ||
== Builds == | == Builds == | ||
This section should contain links for builds with the feature: | This section should contain links for builds with the feature: | ||
* Links for Nightly builds - available since November 26th 2016 - [http://archive.mozilla.org/pub/firefox/nightly/2016/11/2016-11-26-03-02-07-mozilla-central/ link] | * Links for Nightly builds - available since November 26th 2016 - [http://archive.mozilla.org/pub/firefox/nightly/2016/11/2016-11-26-03-02-07-mozilla-central/ link] | ||
* Links for Beta builds - Fx55 | |||
* Links for Beta builds | |||
== Test Execution Schedule == | == Test Execution Schedule == | ||
Line 122: | Line 142: | ||
|- | |- | ||
| Start project | | Start project | ||
|style="text-align:center;" | 02. | |style="text-align:center;" | 02.23.2017 || | ||
|- | |- | ||
| Study documentation/specs received from developers | | Study documentation/specs received from developers | ||
|style="text-align:center;" | | |style="text-align:center;" | 02.28.2017 || 02.29.2017 | ||
|- | |- | ||
| QA - Test plan creation | | QA - Test plan creation | ||
|style="text-align:center;" | 02. | |style="text-align:center;" | 02.23.2017 || 02.24.2017 | ||
|- | |- | ||
| QA - Test cases/Env preparation | | QA - Test cases/Env preparation | ||
|style="text-align:center;" | || | |style="text-align:center;" | 02.28.2017 || 03.30.2017 | ||
|- | |- | ||
| QA - Nightly Testing | | QA - Nightly Testing | ||
|style="text-align:center;" | | |style="text-align:center;" | 02.28.2017 || 05.31.2017 | ||
|- | |- | ||
| QA - Beta Testing | | QA - Beta Testing | ||
Line 163: | Line 180: | ||
= Status = | = Status = | ||
== Overview == | == Overview == | ||
'''Use a separate content process for file:// URLs''' feature first landed in Nightly 53 - [http://archive.mozilla.org/pub/firefox/nightly/2016/11/2016-11-26-03-02-07-mozilla-central/ 53.0a1 (2016-11-26)] | |||
The feature remained in Nightly for the 54 and 55 cycle. | |||
= References = | = References = | ||
* | * {{Bug|1147911}} - Use a separate content process for file:// URLs | ||
= Testcases = | = Testcases = | ||
== | Full testsuites can be found on TestRail at [https://testrail.stage.mozaws.net/index.php?/suites/view/576&group_by=cases:section_id&group_order=asc link] or in the [https://wiki.mozilla.org/images/4/42/Separate_content_process_for_file_URLs_-EngQA-_-_TestRail.pdf PDF doc]. | ||
== Test Areas == | == Test Areas == | ||
{| class="wikitable" style="width:80%" | {| class="wikitable" style="width:80%" | ||
Line 187: | Line 194: | ||
|- | |- | ||
| Private Window | | Private Window | ||
|style="text-align:center;" | || | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Multi-Process Enabled | | Multi-Process Enabled | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Multi-process Disabled | | Multi-process Disabled | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Theme (high contrast) | | Theme (high contrast) | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| '''UI''' | | '''UI''' | ||
Line 202: | Line 209: | ||
|- | |- | ||
| Mouse-only operation | | Mouse-only operation | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes|| | ||
|- | |- | ||
| Keyboard-only operation | | Keyboard-only operation | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Display (HiDPI) | | Display (HiDPI) | ||
|style="text-align:center;" | || | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Interraction (scroll, zoom) | | Interraction (scroll, zoom) | ||
|style="text-align:center;" | || | |style="text-align:center;" | No || | ||
|- | |- | ||
| Usable with a screen reader | | Usable with a screen reader | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || e.g. with NVDA | ||
|- | |- | ||
| Usability and/or discoverability testing | | Usability and/or discoverability testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || Is this feature user friendly | ||
|- | |- | ||
| RTL build testing | | RTL build testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| '''Help/Support''' | | '''Help/Support''' | ||
Line 226: | Line 233: | ||
|- | |- | ||
| Help/support interface required | | Help/support interface required | ||
|style="text-align:center;" | | |style="text-align:center;" | No || Make sure link to support/help page exist and is easy reachable. | ||
|- | |- | ||
| Support documents planned(written) | | Support documents planned(written) | ||
|style="text-align:center;" | | |style="text-align:center;" | No || Make sure support documents are written and are correct. | ||
|- | |- | ||
Line 236: | Line 243: | ||
|- | |- | ||
| Feature upgrades/downgrades data as expected | | Feature upgrades/downgrades data as expected | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || | ||
|- | |- | ||
| Does sync work across upgrades | | Does sync work across upgrades | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Requires install testing | | Requires install testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || separate feature/application installation needed (not only Firefox) | ||
|- | |- | ||
| Affects first-run or onboarding | | Affects first-run or onboarding | ||
|style="text-align:center;" | | |style="text-align:center;" | No || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned. | ||
|- | |- | ||
| Does this affect partner builds? Partner build testing | | Does this affect partner builds? Partner build testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || yes/no options, add comment with details about who will lead testing | ||
|- | |- | ||
Line 255: | Line 262: | ||
|- | |- | ||
| Enterprise administration | | Enterprise administration | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Network proxies/autoconfig | | Network proxies/autoconfig | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| ESR behavior changes | | ESR behavior changes | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Locked preferences | | Locked preferences | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
Line 271: | Line 278: | ||
|- | |- | ||
| Temporary or permanent telemetry monitoring | | Temporary or permanent telemetry monitoring | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || List of error conditions to monitor | ||
|- | |- | ||
| Telemetry correctness testing | | Telemetry correctness testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Server integration testing | | Server integration testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Offline and server failure testing | | Offline and server failure testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Load testing | | Load testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
Line 290: | Line 297: | ||
|- | |- | ||
| Addon API required? | | Addon API required? | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Comprehensive API testing | | Comprehensive API testing | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Permissions | | Permissions | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Testing with existing/popular addons | | Testing with existing/popular addons | ||
|style="text-align:center;" | | |style="text-align:center;" | Yes || | ||
|- | |- | ||
Line 316: | Line 323: | ||
|- | |- | ||
| ''' Web Compatibility ''' | | ''' Web Compatibility ''' | ||
|| | || || depends on the feature | ||
|- | |- | ||
| Testing against target sites | | Testing against target sites | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Survey of many sites for compatibility | | Survey of many sites for compatibility | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
Line 329: | Line 336: | ||
|- | |- | ||
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. | | Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS | | Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|- | |- | ||
| Interaction of this feature with other browser features | | Interaction of this feature with other browser features | ||
|style="text-align:center;" | | |style="text-align:center;" | No || | ||
|} | |} | ||
== Test suite == | == Test suite == | ||
Full Test suite - Link with the gdoc, follow the format from [https:// | Full Test suite - Link with the gdoc, follow the format from [https://testrail.stage.mozaws.net/index.php?/suites/view/576&group_by=cases:section_id&group_order=asc link] | ||
= Bug Work = | = Bug Work = | ||
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1147911 1147911 - Use a separate content process for file:// URLs] | |||
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"> | <div class="toccolours mw-collapsible mw-collapsed" style="width:auto"> | ||
====== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=1147911 1147911 ] )====== | |||
<div class="mw-collapsible-content"> | <div class="mw-collapsible-content"> | ||
{ | <bugzilla> | ||
{ | |||
"blocks":[1147911], | |||
"include_fields": "id, priority, component, assigned_to, summary, status, target_milestone" | |||
} | |||
</bugzilla> | |||
</div> | |||
</div></div> | </div> | ||
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"> | <div class="toccolours mw-collapsible mw-collapsed" style="width:auto"> | ||
====== Bug fix verification ====== | |||
====== | |||
<div class="mw-collapsible-content"> | <div class="mw-collapsible-content"> | ||
<bugzilla> | |||
{ | |||
"blocks":[1147911], | |||
"resolution":"FIXED", | |||
"include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone" | |||
} | |||
</bugzilla> | |||
</div> | </div> | ||
</div> | </div> | ||
= Sign off = | |||
== Criteria for Mid Nightly == | |||
Check list | |||
* All the smoke tests (designed to ensure the main functionality is working as expected) are executed. | |||
== Criteria for Pre-Beta Release and Pre-Release == | |||
== Criteria == | |||
Check list | Check list | ||
* All test cases should be executed | * All test cases should be executed | ||
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA) | * All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA) | ||
== Results == | == Results == | ||
'''Nightly testing'''<br /> | '''Mid Nightly testing'''<br /> | ||
Testing performed on folowing OSes<br /> | |||
* | * Windows 7 <br /> | ||
* | * Ubuntu 16.04<br /> | ||
* | * Mac OS X 10.12 <br /> | ||
''' | Link for the tests run - [https://testrail.stage.mozaws.net/index.php?/plans/view/2907 Separate content process for file:// URLs [EngQA<nowiki>]</nowiki> - Mid Nightly 55.0a1] | ||
*Link for the tests run | '''Pre-Beta testing'''<br /> | ||
Testing performed on folowing OSes<br /> | |||
* Windows 10 x64 <br /> | |||
* Ubuntu 16.04<br /> | |||
* Mac OS X 10.12 <br /> | |||
Link for the tests run - [https://testrail.stage.mozaws.net/index.php?/plans/view/2907 Separate content process for file:// URLs - Pre-Beta Sign off (Fx55)] <br /> | |||
Links for the test report - [https://testrail.stage.mozaws.net/index.php?/reports/view/352 link] | |||
== Checklist == | == Checklist == | ||
{| class="wikitable | {| class="wikitable" | ||
|- | |- | ||
! Exit Criteria !! Status !! Notes/Details | ! Exit Criteria !! Status !! Notes/Details | ||
|- | |- | ||
| Testing Prerequisites (specs, use cases) | | Testing Prerequisites (specs, use cases) | ||
| style="text-align:center;" | | | style="text-align:center;" | {{mdone|}} | ||
| style="text-align:center;" | | | style="text-align:center;" | | ||
|- | |- | ||
| Testing Infrastructure setup | | Testing Infrastructure setup | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} || | ||
|- | |- | ||
| Test Plan Creation | | Test Plan Creation | ||
| style="text-align:center;" | | | style="text-align:center;" | {{mdone|}} || | ||
|- | |- | ||
| Test Cases Creation | | Test Cases Creation | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} || | ||
|- | |- | ||
| Full Functional Tests Execution | | Full Functional Tests Execution | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} | ||
|style="text-align:center;" | | |style="text-align:center;" | | ||
|- | |- | ||
| Automation Coverage | | Automation Coverage | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} || | ||
toolkit/mozapps/extensions/test/browser/browser_file_xpi_no_process_switch.js <br /> | |||
dom/html/test/browser_form_post_from_file_to_http.js <br /> | |||
browser/base/content/test/tabs/browser_new_web_tab_in_file_process_pref.js <br /> | |||
browser/base/content/test/tabs/browser_viewsource_of_data_URI_in_file_process.js <br /> | |||
browser/base/content/test/general/browser_new_http_window_opened_from_file_tab.js <br /> | |||
browser/modules/test/xpcshell/test_E10SUtils_nested_URIs.js <br /> | |||
browser/base/content/test/tabs/browser_allow_process_switches_despite_related_browser.js <br /> | |||
browser/base/content/test/tabs/browser_reload_deleted_file.js <br /> | |||
browser/base/content/test/tabs/browser_opened_file_tab_navigated_to_web.js <br /> | |||
|- | |- | ||
| Performance Testing | | Performance Testing | ||
|style="text-align:center;" | | |style="text-align:center;" | N/A || | ||
|- | |- | ||
| All Defects Logged || || | | All Defects Logged | ||
|style="text-align:center;" | {{mdone|}} || | |||
|- | |- | ||
| Critical/Blockers Fixed and Verified || || | | Critical/Blockers Fixed and Verified | ||
|style="text-align:center;" | {{mdone|}} || | |||
|- | |- | ||
| Metrics/Telemetry | | Metrics/Telemetry | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} || | ||
|- | |- | ||
| QA | | QA mid-Nightly Signoff | ||
|style="text-align:center;" | | |style="text-align:center;" | {{mdone|}} | ||
| | |style="text-align:center;" | [[https://wiki.mozilla.org/images/7/7a/-desktop--feature-_Use_a_separate_content_process_for_local_files_-_mid_Nightly_sign_off_%28YELLOW%29.pdf Email sent in 5/16/2017]] | ||
|style="text-align:center;" | | |||
|- | |- | ||
| QA Signoff | | QA pre-Beta Signoff|| | ||
|style="text-align:center;"| Email to be sent | |style="text-align:center;"| Email to be sent | ||
|- | |- | ||
Line 438: | Line 465: | ||
|style="text-align:center;" | || | |style="text-align:center;" | || | ||
|- | |- | ||
| QA | | QA pre-Release Signoff || | ||
|style="text-align:center;" | Email to be sent | |style="text-align:center;" | Email to be sent | ||
|} | |} |