Auto-tools/Automation Development/Meetings/130304: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(19 intermediate revisions by 6 users not shown)
Line 9: Line 9:
= General =
= General =
* Attendees:  
* Attendees:  
**  
** Daniela, Mihaelav, Andreea, Rob,  Otilia, Joel, Mario, Adrian, Henrik, Anthony, Dave, Clint
* Chair: Rob Wood
* Chair: Rob Wood
** Welcome! :-) http://tinyurl.com/btcdndm
** Welcome! :-) http://tinyurl.com/btcdndm
Line 16: Line 16:
= Last Weeks Action Items =
= Last Weeks Action Items =
* {{prev|}} Joel will talk to Tony and will look into what robocop tests we could  add for performance/stress - currently covering more of the basic  functionality is more important (email dialog started)
* {{prev|}} Joel will talk to Tony and will look into what robocop tests we could  add for performance/stress - currently covering more of the basic  functionality is more important (email dialog started)
* {{prev|}} Adrian will get access to the tryserver - Jeol will need to log a bug for that (bug 844892 filed)
* {{prev|}} Adrian will get access to the tryserver - Joel filed bug 844892. Adrian added the necessary information in the bug and is awaiting confirmation or requests for other information
* {{new|}} [Henrik] - Disable flash on OSX 10.6
* {{done|}} [Henrik] - Disable flash on OSX 10.6
* {{new|}}[Henrik] - Start a thread in the mailing list for additional 'Ask an Expert' session
** That fixed the log spam on those machines
* {{new|}}[Andrei] - amed our style guide to include a section about Bug Fixes vs Style changes
* {{done|}}[Henrik] - Start a thread in the mailing list for additional 'Ask an Expert' session
** https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.automation/OencYa9k8mQ
* {{new|}}[Andrei] - amend our style guide to include a section about Bug Fixes vs Style changes
* {{new|}}[Andreea] - File a bug against Automation bot to add the functionality
* {{new|}}[Andreea] - File a bug against Automation bot to add the functionality
* {{new|}}[Andreea] - Look into VPN issues (unable to recieve email while connected)
** [http://automatedtester.github.com/automation-services-bot/ Source] is redirecting me
* {{done|}}[Andreea] - Look into VPN issues (unable to receive email while connected)
* {{new|}}[Henrik] - Extra item to the issue for mozmill-dashboard. adding support for Metro tests
* {{new|}}[Henrik] - Extra item to the issue for mozmill-dashboard. adding support for Metro tests
** Not sure what this is about?
* {{done|}} [Anthony] - Get back to Clint about Desktop team's automation priorities
** chemspill regression tests > metro desktop tests > new feature tests


= Highlights =
= Highlights =
* You can find all blockers here: http://bit.ly/Tph0NY
* You can find all blockers here: http://bit.ly/Tph0NY
* [henrik] We are still green for Mozmill tests since 2 weeks now!!
* [henrik] We stopped running tests for ESR10 builds in Mozmill CI


= Roundtable =
= Roundtable =
Add here any issues or topics you want to discuss with the team
* Do we want to open up the weekly etherpad for contributors too? This would be a good step to let others see on what we are working on. Otherwise we might considering blog posts?
*


= Project Updates =
= Project Updates =
Line 35: Line 42:
== Gaia UI (Dave) ==
== Gaia UI (Dave) ==
* Goals progress:
* Goals progress:
**
** [rwood] Goal: Write stability/long running/endurance tests and graph their performance over time [ON TRACK]
* [rwood] Developing long-running GaiaUI tests ({{bug|843083}}). Last week completed tests for airplane mode, SMS, video playback, browser, and edit/delete events. For a complete list please see the [https://ateam.etherpad.mozilla.org/GaiaUIStress GaiaUI Stress Test etherpad].
*
*


== WebAPI (Rob) ==
== WebAPI (Rob) ==
* Goals progress:
* Goals progress:
**  
** [rwood] This goal was completed last quarter except for two tests still under review/need minor updates
*
* [rwood] Minor changes required for two Screen Orientation tests, once submitted and they receive an r+ this section can be removed from the meeting agenda.


== WebRTC (Henrik) ==
== WebRTC (Henrik) ==
* Goals progress:
* Goals progress:
**  
** Waiting for the new framework to be implemented to get started
*
* {{bug|837458}} Cleaning up the framework for PeerConnection tests


= Softvision Updates =
= Softvision Updates =
Line 55: Line 63:
** {{bug|843553}} - Test failure "Checkbox ID: acceptCookies could not be checked/unchecked" in /testPreferences/testDisableCookies.js
** {{bug|843553}} - Test failure "Checkbox ID: acceptCookies could not be checked/unchecked" in /testPreferences/testDisableCookies.js
** [https://github.com/mozilla/gaia-ui-tests/issues/409 Gaia Issue 409]
** [https://github.com/mozilla/gaia-ui-tests/issues/409 Gaia Issue 409]
** {{bug|807251}} - Test failure "Suggestions are different - got 'false'" in testSearch/testSearchSuggestions.js


* Blocked bugs:
* Blocked bugs:
Line 60: Line 69:
** {{bug|763159}} - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js  
** {{bug|763159}} - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js  


* Waiting for review / feedback / Checkin needed
* Waiting for review / feedback / Need info
** {{bug|489474}} - Write automated tests for user generated video controls
** {{bug|489474}} - Write automated tests for user generated video controls
** {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend
** {{bug|826251}} - Make testPreferences/testPreferredLanguage.js language independend
** {{bug|842571}} - Update the process start directory (XCurProcD) in software-update.js library
** {{bug|842571}} - Update the process start directory (XCurProcD) in software-update.js library
** {{bug|827752}} - Create Mozmill test to verify that Flash content respects Firefox's private browsing


* Working on:
* Working on:
Line 70: Line 80:
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies
** {{bug|732353}} - Disable all Discovery Pane tests due to unpredictable web dependencies
** {{bug|840022}} - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
** {{bug|840022}} - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
** {{bug|839996}} - Add event to wait for before importing the default bookmarks through importFromURL function
** {{bug|827752}} - Create Mozmill test to verify that Flash content respects Firefox's private browsing


== Mobile (Robocop) ==
== Mobile (Robocop) ==
*
*'''Automation testing status'''
* Progress on [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdDJBT2dlbVJwUk9PSy1RbHo4WVNiUGc#gid=8 Robocop Automation]
* Made a first look estimation of how many Moztrap tests can be covered using Robocop - please see the [https://docs.google.com/spreadsheet/ccc?key=0Ak8KPfUd1-5udC1FRXBHMlA2MkJLOEp0NFkxTnBtS1E#gid=1 Google spreadsheet]
* Bugs with work in progress:
** {{Bug|820276}} - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests
** {{Bug|822259}} - Robocop: Add test for Find in Page feature - blocked by [https://bugzilla.mozilla.org/show_bug.cgi?id=828481 bug 828481]
** {{Bug|820859}} - Robocop: Add test for 'Clear Private Data'
** {{Bug|830834}} - Robocop: Add test for 'Show Search Suggestions' feature
** {{Bug|833832}} - Robocop: Add test for Import from Android feature
** {{Bug|846257}} - Robocop: Extend 'Web Content Context Menu' test to cover the context menu options for images
** {{Bug|846340}} - Robocop: Add test for 'Clear Site Settings'
** {{Bug|838596}} - Robocop: Add test for 'Master Password' feature
** {{Bug|846296}} - Robocop: Add test for 'Add Search Engine' feature
* '''Issues/Questions'''
** There were a few patches uploaded last week but unfortunately they seem to fail on the tegraboards. We are working to fix those.


= Personal Status =
= Personal Status =
Line 83: Line 109:


== New action items ==
== New action items ==
*
* {{prev|}} [Andrei] - amend our style guide to include a section about Bug Fixes vs Style changes
* {{prev|}} [Andreea] - File a bug against Automation bot to add the functionality
* {{new|}} [Henrik] - Look into how to run tests against Metro and what will be needed to display results in the dashboard
* {{new|}} Anthony - Ensure any Metro tests are suitably tagged with in-testsuite?

Latest revision as of 19:38, 4 March 2013

previous meeting | Meetings | next meeting »

Dial in

 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:  650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #         1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:    irc://irc.mozilla.org:6697/#automation

General

  • Attendees:
    • Daniela, Mihaelav, Andreea, Rob, Otilia, Joel, Mario, Adrian, Henrik, Anthony, Dave, Clint
  • Chair: Rob Wood
  • Notes: Dave Hunt

Last Weeks Action Items

  • [CARRY OVER] Joel will talk to Tony and will look into what robocop tests we could add for performance/stress - currently covering more of the basic functionality is more important (email dialog started)
  • [CARRY OVER] Adrian will get access to the tryserver - Joel filed bug 844892. Adrian added the necessary information in the bug and is awaiting confirmation or requests for other information
  • [DONE] [Henrik] - Disable flash on OSX 10.6
    • That fixed the log spam on those machines
  • [DONE] [Henrik] - Start a thread in the mailing list for additional 'Ask an Expert' session
  • [NEW] [Andrei] - amend our style guide to include a section about Bug Fixes vs Style changes
  • [NEW] [Andreea] - File a bug against Automation bot to add the functionality
  • [DONE] [Andreea] - Look into VPN issues (unable to receive email while connected)
  • [NEW] [Henrik] - Extra item to the issue for mozmill-dashboard. adding support for Metro tests
    • Not sure what this is about?
  • [DONE] [Anthony] - Get back to Clint about Desktop team's automation priorities
    • chemspill regression tests > metro desktop tests > new feature tests

Highlights

  • You can find all blockers here: http://bit.ly/Tph0NY
  • [henrik] We are still green for Mozmill tests since 2 weeks now!!
  • [henrik] We stopped running tests for ESR10 builds in Mozmill CI

Roundtable

  • Do we want to open up the weekly etherpad for contributors too? This would be a good step to let others see on what we are working on. Otherwise we might considering blog posts?

Project Updates

Gaia UI (Dave)

  • Goals progress:
    • [rwood] Goal: Write stability/long running/endurance tests and graph their performance over time [ON TRACK]
  • [rwood] Developing long-running GaiaUI tests (bug 843083). Last week completed tests for airplane mode, SMS, video playback, browser, and edit/delete events. For a complete list please see the GaiaUI Stress Test etherpad.

WebAPI (Rob)

  • Goals progress:
    • [rwood] This goal was completed last quarter except for two tests still under review/need minor updates
  • [rwood] Minor changes required for two Screen Orientation tests, once submitted and they receive an r+ this section can be removed from the meeting agenda.

WebRTC (Henrik)

  • Goals progress:
    • Waiting for the new framework to be implemented to get started
  • bug 837458 Cleaning up the framework for PeerConnection tests

Softvision Updates

  • Fixed:
    • bug 781547 - Failure in "/testBookmarks_OpenAllInTabs/test1.js" with "Default bookmarks have been imported"
    • bug 843553 - Test failure "Checkbox ID: acceptCookies could not be checked/unchecked" in /testPreferences/testDisableCookies.js
    • Gaia Issue 409
    • bug 807251 - Test failure "Suggestions are different - got 'false'" in testSearch/testSearchSuggestions.js
  • Blocked bugs:
    • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted - blocked by bug 804952 - Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check
    • bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
  • Waiting for review / feedback / Need info
    • bug 489474 - Write automated tests for user generated video controls
    • bug 826251 - Make testPreferences/testPreferredLanguage.js language independend
    • bug 842571 - Update the process start directory (XCurProcD) in software-update.js library
    • bug 827752 - Create Mozmill test to verify that Flash content respects Firefox's private browsing
  • Working on:
    • bug 800872 - Test failure "Window number '1' has been opened" in endurance//testTabbedBrowsing_OpenNewWindow/test1.js
    • bug 834163 - Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
    • bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
    • bug 840022 - Test failure "The forward button has been made visible for the 1 page" in testToolbars/testBackForwardButtons.js
    • bug 839996 - Add event to wait for before importing the default bookmarks through importFromURL function
    • bug 827752 - Create Mozmill test to verify that Flash content respects Firefox's private browsing

Mobile (Robocop)

  • Automation testing status
  • Progress on Robocop Automation
  • Made a first look estimation of how many Moztrap tests can be covered using Robocop - please see the Google spreadsheet
  • Bugs with work in progress:
    • bug 820276 - Robocop: Make editBookmark and checkBookmarkEdit methods available to other tests
    • bug 822259 - Robocop: Add test for Find in Page feature - blocked by bug 828481
    • bug 820859 - Robocop: Add test for 'Clear Private Data'
    • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
    • bug 833832 - Robocop: Add test for Import from Android feature
    • bug 846257 - Robocop: Extend 'Web Content Context Menu' test to cover the context menu options for images
    • bug 846340 - Robocop: Add test for 'Clear Site Settings'
    • bug 838596 - Robocop: Add test for 'Master Password' feature
    • bug 846296 - Robocop: Add test for 'Add Search Engine' feature
  • Issues/Questions
    • There were a few patches uploaded last week but unfortunately they seem to fail on the tegraboards. We are working to fix those.

Personal Status

For the personal status please check the weekly status updates:

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

New action items

  • [CARRY OVER] [Andrei] - amend our style guide to include a section about Bug Fixes vs Style changes
  • [CARRY OVER] [Andreea] - File a bug against Automation bot to add the functionality
  • [NEW] [Henrik] - Look into how to run tests against Metro and what will be needed to display results in the dashboard
  • [NEW] Anthony - Ensure any Metro tests are suitably tagged with in-testsuite?
Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/Automation_Development/Meetings/130304&oldid=535573"