QA/Automation/Meetings/150119: Difference between revisions

From MozillaWiki
< QA‎ | Automation‎ | Meetings
Jump to navigation Jump to search
mNo edit summary
(→‎Testday Bot (Barbara): 150119 updates)
 
(17 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<small>[[QA/Automation/Meetings/150112|previous meeting]] | [[QA/Automation/Meetings|Meetings]] | [[QA/Automation/Meetings/150126|next meeting &raquo;]]</small>
<small>[[QA/Automation/Meetings/150112|previous meeting]] | [[QA/Automation/Meetings|Meetings]] | [[QA/Automation/Meetings/150126|next meeting &raquo;]]</small>
= Dial in =
= Dial in =
{{:QA/Automation/Meeting_Details}}
{{:QA/Automation/Meeting_Details}}
= General =
= General =
* Attendees:
* Attendees: Mihaela, Andreea, Danela, Cosmin, Teodor, Henrik
* Chair: Andreea
* Chair: Andreea
* Notes: Mihaela
* Notes: Mihaela
= Last Weeks Action Items =
= Last Weeks Action Items =
* {{new|}} Barbara - to talk to Anthony about the testday bot, if he wants to use it for next test day
* {{new|}} Barbara - to talk to Anthony about the testday bot, if he wants to use it for next test day
* {{ok|}} Andreea - provide statistics about WSAEINVAL dc
** henrik discussed with Anthony who is happy with this and wil start to use it this Friday   
 
* {{done|}} Andreea - provide statistics about WSAEINVAL dc
** https://docs.google.com/spreadsheets/d/1f-ktCXGse8KlutQbMbr5kbq-BJaKnL9XdbWb4Wvst0U/edit?usp=sharing
** Statistics since December 1st
** Henrik will have a look to see with which machines to start
** someone to trigger some tests in loops to see which fails the most
** we could update the document by weekly basis; it takes ~ 2 hours to check manually in the emails
*** maybe we need to find another way to get those statistics
= Highlights =
= Highlights =
* [henrik] Firefox Automation report – week 47/48 2014: http://www.hskupin.info/2015/01/14/firefox-automation-report-week-47-48-2014/
* [henrik] Firefox Automation report – week 47/48 2014: http://www.hskupin.info/2015/01/14/firefox-automation-report-week-47-48-2014/
* [henrik] New Bugzilla component ("Mozilla QA -> Firefox UI Tests") for Marionette tests has been created (Bug 1120643)
* [henrik] New Bugzilla component ("Mozilla QA -> Firefox UI Tests") for Marionette tests has been created (Bug 1120643)
 
* [henrik] mozmill-crowd 0.3 has been released: https://addons.mozilla.org/en-US/firefox/addon/mozmill-crowd/
* [Ioana Chiorean] - text only as I can't attend the meeting
** Please consider to share from your work in the next community calls
** Next one will be TUE, January 27t. Etherpad:  https://etherpad.mozilla.org/Quality-Automation-Call-2015-01-27
* ujjwal had some contributing work done
= Blockers =
= Blockers =
* List of blockers (http://bit.ly/Tph0NY):
* List of blockers (http://bit.ly/Tph0NY):
** [07/10] Bug 735184 - RFE: Create Pulse notifications for update channel activities (goal for Q1/2015)
** [07/10] Bug 735184 - RFE: Create Pulse notifications for update channel activities (goal for Q1/2015)
*** no updates
** [09/03] Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps)
** [09/03] Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps)
*** no updates
** [10/29] Bug 1066493 - TPS failure:  Error: signIn() failed with: "error.sync.reason.serverMaintenance" (would need fix from DB vendor)
** [10/29] Bug 1066493 - TPS failure:  Error: signIn() failed with: "error.sync.reason.serverMaintenance" (would need fix from DB vendor)
*** no updates
** [11/05] Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals
** [11/05] Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals
*** no issues in the last weeks, decrease the priority
** [12/03] Bug 1107405 - Doorhanger notifications don't show up only before Firefox closes
** [12/03] Bug 1107405 - Doorhanger notifications don't show up only before Firefox closes
** <strike>[01/08] Bug 1119146 - Reinstall OSX 10.6.8 on mm-osx-106-4 and mm-osx-106-2, mm-osx-107-2</strike>
*** no updates, we will need to increase the priority
** [01/08] Bug 1119146 - Reinstall OSX 10.6.8 on mm-osx-106-4 and mm-osx-106-2, mm-osx-107-2
*** nearly done, waiting for 106-4
** <strike>[01/12] Bug 1105759 - EV/OV status of website is not displayed when using publicproxy1.scl3.mozilla.com due to hard limitation to mozqa.com</strike>
** <strike>[01/12] Bug 1105759 - EV/OV status of website is not displayed when using publicproxy1.scl3.mozilla.com due to hard limitation to mozqa.com</strike>
 
*** fixed
= Roundtable =
= Roundtable =
* TBD
* TBD
= Goals =
= Goals =
* https://wiki.mozilla.org/QA/Automation#Goals
* https://wiki.mozilla.org/QA/Automation#Goals
== Framework and Tools (Henrik) ==
== Framework and Tools (Henrik) ==
=== Convert first sets of Mozmill tests to Marionette ({{bug|1119715}}) ===
=== Convert first sets of Mozmill tests to Marionette ({{bug|1119715}}) ===
* TBD
* Renamed repository to firefox-ui-tests ([https://github.com/mozilla/firefox-greenlight-tests/issues/44 issue 44])
 
** name is independent from framework
* Refactored code into sub-packages (for harness, puppeteer, and tests) ({{bug|1122015}})
** if we need to take out the packages we don't need to update the other ones
* Will add workaround for handling of Windows and Tabs for now due to lots of dependencies on observer and event handling
** Henrik will implement a new API for observers and notifications which will help converting tests to Marionette
=== Run Marionette tests in Mozmill CI ({{bug|1119725}}) ===
=== Run Marionette tests in Mozmill CI ({{bug|1119725}}) ===
* TBD
* Will be started once handling of windows and tabs is done
 
** need to work on the harness after that
=== Create Pulse notifications for update channel activities ({{bug|735184}}) ===
=== Create Pulse notifications for update channel activities ({{bug|735184}}) ===
* TBD
* Collected all the build notifications which needs to be investigated now
** updated the listener


== Test Automation (Softvision) ==
== Test Automation (Softvision) ==
=== Creation of automated tests for important regressions ===
=== Creation of automated tests for important regressions ===
* TBD
* Fixed:
 
** {{bug|1110677}} - Add automated test for "Change newtab page appearance"
** {{bug|1098351}} - Add new automated test for "Override mixed script content blocking"
* In progress:
** {{bug|1109140}} - Create a remote Mozmill test for "What is this page?" links in "about:newtab" | Blocked by {{bug|1117745}} - Status: In Review
** {{bug|1110187}} - Create a remote Mozmill test for "Sponsored" tiles in "about:newtab" | Blocked by {{bug|1117745}} - Status: In Review
** {{bug|1037416}} - Add mozmill test for Developer Tools keyboard shortcuts
* This new Sprint starting today we plan on finishing these 3 in progress above and we have 2 more added.
= Project Updates =
= Project Updates =
== Mozmill Tests (Softvision) ==
== Mozmill Tests (Softvision) ==
* TBD
* 12 fixed bugs:
 
* {{bug|1108948}} - Test failure 'Addon is no longer marked for uninstall' in /testAddons/testEnableDisableUndoUninstall.js
* {{bug|1100953}} - Test failure "Tab with URL 'http://localhost:43336/layout/mozilla.html' has been opened" in testRestorePreviousSession/test2.js
* {{bug|1112527}} - Failure in saving screenshots for l10n test results: "Not enough arguments [nsIWebBrowserPersist.saveURI]"
* {{bug|1122530}} - Test failure "Tab has been closed" in /ide@seleniumhq.org/tests/testCommands/testAlert/testAssertAlertFails.js
* {{bug|1112132}} - Functional test 'testAwesomeBar/testAccessLocationBar.js' is not doing what it's supposed to
* {{bug|1120382}} - Test failures in /testSecurity/testSSLDisabledErrorPage.js | testDisableSSL
* {{bug|1120411}} - Remove unnecessary "+" characters from browser.js
* {{bug|1085328}} - Test failure "The auto-complete result is a bookmark - 'favicon' should contain 'bookmark'" in /testAwesomeBar/testSuggestBookmarks.js
* {{bug|1117743}} - Merge mozmill-tests branches for Firefox 35 release
* {{bug|1108999}} - Refactor testAddons_installFromFTP from multiple files into a single file
* {{bug|1109026}} - Refactor testPreferences_masterPassword from multiple files into a single file
* {{bug|1036294}} - Refactor restart tests from multiple files into a single file
== Mozmill (Henrik) ==
== Mozmill (Henrik) ==
* TBD
* No updates
 
== TPS (Henrik) ==
== TPS (Henrik) ==
* TBD
* No updates
 
== Testday Bot (Barbara) ==
== Testday Bot (Barbara) ==
* TBD
* outstanding P1/P2 issues
** stats reporting: use mozqa.com
** provide information on joining the community
* in progress, close to merge
** https://github.com/mozilla/testdaybot/pull/48 - Save and reload Test Day data across bot restarts
* done
** https://github.com/mozilla/testdaybot/pull/34 - Update channel topic when Test Day starts and ends
** https://github.com/mozilla/testdaybot/pull/38 - Advertise Test Day
** https://github.com/mozilla/testdaybot/pull/45 - update node-irc to current version 0.3.7
** https://github.com/mozilla/testdaybot/pull/33 - Add user verification for admin commands
** https://github.com/mozilla/testdaybot/pull/36 - Add command to show and notify Test Day helpers
** https://github.com/mozilla/testdaybot/pull/40 - Add start/end/topic to stats output
** https://github.com/mozilla/testdaybot/pull/41 - Add :schedule command to provide Test Day schedule
** https://github.com/mozilla/testdaybot/pull/42 - update README.md


= Softvision Updates =
= Softvision Updates =
* Detailed progress information from last week: https://groups.google.com/forum/#!topic/mozilla.dev.automation/%ID%
* Detailed progress information from last week: https://groups.google.com/forum/#!topic/mozilla.dev.automation/brTGzWy8icM
 
= Personal Status =
= Personal Status =
For the personal status please check the weekly status updates:
For the personal status please check the weekly status updates:
* [https://mozqa.etherpad.mozilla.org/mozauto-150119 150119 - 150125]
* [https://mozqa.etherpad.mozilla.org/mozauto-150119 150119 - 150125]
= Meeting Notes =
= Meeting Notes =
The meeting notes for this meeting can be read in the following etherpad:  
The meeting notes for this meeting can be read in the following etherpad:  
* http://etherpad.mozilla.com:9000/automation-meeting-20150119
* http://etherpad.mozilla.com:9000/automation-meeting-20150119
If you missed the meeting you can also watch it in our [https://vreplay.mozilla.com/replay/login.html?sortBy=date&viewCount=1&currentPage=1&groupBy=combsearchFilter%3DFirefox+automation+meeting&roomFilter=&usernameFilter=&searchFilter=Firefox+AUtomation&usernameFullFilter=&myManager=-1&adminManager=0&webCast=0&command=&recId=&auxMessage=&auxMessage1=&lang=en&langChanged=&tenantFilter=&securityTab= archive].
If you missed the meeting you can also watch it in our [https://vreplay.mozilla.com/replay/login.html?sortBy=date&viewCount=1&currentPage=1&groupBy=combsearchFilter%3DFirefox+automation+meeting&roomFilter=&usernameFilter=&searchFilter=Firefox+AUtomation&usernameFullFilter=&myManager=-1&adminManager=0&webCast=0&command=&recId=&auxMessage=&auxMessage1=&lang=en&langChanged=&tenantFilter=&securityTab= archive].
==Action items==
==Action items==
* {{new|}} %name%: %task%
* {{new|}} [Softvision] Trigger some tests in loops to see which machines are most affected by the failures (WSAEINVAL or the most affected one...)
 
** have a simple test case which reproduces the problem and run it with Mozmill 2.1 which gives more info when FF crashes or disconnects
 
** use 2.0.9 to see get the test then use 2.1 to get more info
* Chair: Teodor
* Notes: Cosmin
[[Category:Wikipage templates]]
[[Category:Wikipage templates]]

Latest revision as of 17:37, 19 January 2015

previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

  • Attendees: Mihaela, Andreea, Danela, Cosmin, Teodor, Henrik
  • Chair: Andreea
  • Notes: Mihaela

Last Weeks Action Items

  • [NEW] Barbara - to talk to Anthony about the testday bot, if he wants to use it for next test day
    • henrik discussed with Anthony who is happy with this and wil start to use it this Friday
  • [DONE] Andreea - provide statistics about WSAEINVAL dc

Highlights

Blockers

  • List of blockers (http://bit.ly/Tph0NY):
    • [07/10] Bug 735184 - RFE: Create Pulse notifications for update channel activities (goal for Q1/2015)
      • no updates
    • [09/03] Bug 997230 - Setup Mac Mini for DeployStudio needed by PuppetAgain (needs action RelOps)
      • no updates
    • [10/29] Bug 1066493 - TPS failure: Error: signIn() failed with: "error.sync.reason.serverMaintenance" (would need fix from DB vendor)
      • no updates
    • [11/05] Bug 1083166 - [SCL3] Pulse and maybe other connections are timing out in irregular intervals
      • no issues in the last weeks, decrease the priority
    • [12/03] Bug 1107405 - Doorhanger notifications don't show up only before Firefox closes
      • no updates, we will need to increase the priority
    • [01/08] Bug 1119146 - Reinstall OSX 10.6.8 on mm-osx-106-4 and mm-osx-106-2, mm-osx-107-2
      • nearly done, waiting for 106-4
    • [01/12] Bug 1105759 - EV/OV status of website is not displayed when using publicproxy1.scl3.mozilla.com due to hard limitation to mozqa.com
      • fixed

Roundtable

  • TBD

Goals

Framework and Tools (Henrik)

Convert first sets of Mozmill tests to Marionette (bug 1119715)

  • Renamed repository to firefox-ui-tests (issue 44)
    • name is independent from framework
  • Refactored code into sub-packages (for harness, puppeteer, and tests) (bug 1122015)
    • if we need to take out the packages we don't need to update the other ones
  • Will add workaround for handling of Windows and Tabs for now due to lots of dependencies on observer and event handling
    • Henrik will implement a new API for observers and notifications which will help converting tests to Marionette

Run Marionette tests in Mozmill CI (bug 1119725)

  • Will be started once handling of windows and tabs is done
    • need to work on the harness after that

Create Pulse notifications for update channel activities (bug 735184)

  • Collected all the build notifications which needs to be investigated now
    • updated the listener

Test Automation (Softvision)

Creation of automated tests for important regressions

  • Fixed:
    • bug 1110677 - Add automated test for "Change newtab page appearance"
    • bug 1098351 - Add new automated test for "Override mixed script content blocking"
  • In progress:
    • bug 1109140 - Create a remote Mozmill test for "What is this page?" links in "about:newtab" | Blocked by bug 1117745 - Status: In Review
    • bug 1110187 - Create a remote Mozmill test for "Sponsored" tiles in "about:newtab" | Blocked by bug 1117745 - Status: In Review
    • bug 1037416 - Add mozmill test for Developer Tools keyboard shortcuts
  • This new Sprint starting today we plan on finishing these 3 in progress above and we have 2 more added.

Project Updates

Mozmill Tests (Softvision)

  • 12 fixed bugs:
  • bug 1108948 - Test failure 'Addon is no longer marked for uninstall' in /testAddons/testEnableDisableUndoUninstall.js
  • bug 1100953 - Test failure "Tab with URL 'http://localhost:43336/layout/mozilla.html' has been opened" in testRestorePreviousSession/test2.js
  • bug 1112527 - Failure in saving screenshots for l10n test results: "Not enough arguments [nsIWebBrowserPersist.saveURI]"
  • bug 1122530 - Test failure "Tab has been closed" in /ide@seleniumhq.org/tests/testCommands/testAlert/testAssertAlertFails.js
  • bug 1112132 - Functional test 'testAwesomeBar/testAccessLocationBar.js' is not doing what it's supposed to
  • bug 1120382 - Test failures in /testSecurity/testSSLDisabledErrorPage.js | testDisableSSL
  • bug 1120411 - Remove unnecessary "+" characters from browser.js
  • bug 1085328 - Test failure "The auto-complete result is a bookmark - 'favicon' should contain 'bookmark'" in /testAwesomeBar/testSuggestBookmarks.js
  • bug 1117743 - Merge mozmill-tests branches for Firefox 35 release
  • bug 1108999 - Refactor testAddons_installFromFTP from multiple files into a single file
  • bug 1109026 - Refactor testPreferences_masterPassword from multiple files into a single file
  • bug 1036294 - Refactor restart tests from multiple files into a single file

Mozmill (Henrik)

  • No updates

TPS (Henrik)

  • No updates

Testday Bot (Barbara)

Softvision Updates

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

If you missed the meeting you can also watch it in our archive.

Action items

  • [NEW] [Softvision] Trigger some tests in loops to see which machines are most affected by the failures (WSAEINVAL or the most affected one...)
    • have a simple test case which reproduces the problem and run it with Mozmill 2.1 which gives more info when FF crashes or disconnects
    • use 2.0.9 to see get the test then use 2.1 to get more info
  • Chair: Teodor
  • Notes: Cosmin
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Automation/Meetings/150119&oldid=1048870"