previous meeting | Meetings | next meeting »

Dial in

 # When:     Every Monday at 8:45am PDT/PST
 # Vidyo:    https://v.mozilla.com/flex.html?roomdirect.html&key=PGtLpx3XQGJz
 # Phone:    650-903-0800 or 650-215-1282 x92 Conf# 9654 (US/INTL)
 #           1-800-707-2533 (pin 369) Conf# 9654 (US)
 # IRC:      irc://irc.mozilla.org:6697/#automation
 # IRC logs: http://irclog.gr/#browse/irc.mozilla.org/automation

General

• Attendees:
  • TBD
• Chair: Andreea
• Notes: Daniela

Last Weeks Action Items

• [CARRY OVER] Henrik - Update the goals page
• [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?
• [DONE] Andreea: File bug about the broken elementslib handling with elements - bug 864375

Highlights

• [henrik] mozdownload 1.7 released
• [henrik] Nightly Tester Tools 3.5 released
• [dave] Reminder: I will be on PTO from Thursday 2nd May to Monday 20th May
• [rwood] I will also be on PTO Friday May 3rd to Monday May 20th, inclusive
• New blockers, any updates?
  • [henrik] We are now skipping update tests for build notifications without a previous build id. Those are related to broken previous builds and RelEng can't fix it as of now
  • [henrik] We most likely want to disable update tests for daily localized builds given that RelEng can't fix the issue with broken AUS snippets as of now (bug 858953)
• You can find all blockers here: http://bit.ly/Tph0NY

Roundtable

• [henrik] Why was the old ci-master offline through the whole weekend?
  • I requested to shutdown the old ESX cluster but forgot that our old ci-master is running on a Linux VM inside it. Thanks to Daniela who spotted this today
  • For now the master is reachable via http://release1.qa.mtv1.mozilla.com:8080/
  • I have seen that the minis can connect to the new ci-master. Seems like the network routes have been setup that way. I will create a patch to get the minis into the new master
  • Once IT has finished preparing our new mac minis we will replace them all
• [henrik] Mozillians.org
  • How do we want to name our group: automation development vs. automation?
  • Should we have profiles for each of us with at least the name, picture, and country publicly accessible?
  • I would like to see that groups can also be public so we could reference from our team wiki page. I should possibly file a bug.
• [henrik] Backports of fixes for mozmill-tests
  • What do we want to backport?
    • No style changes or improvements
    • Major changes which correspond to how the automation scripts drive the tests
    • Absolutely fixes for test failures
    • ...

Goals Overview

• https://wiki.mozilla.org/Auto-tools/Automation_Development#Prioritized_Projects

Project Updates

Gaia UI (Dave)

• Goals progress:
• TBD
• We are now able to recover from a failure in B2G desktop client tests on Travis, and at last count have 24 failures (and 36 passes). See the results here.
• Made progress on cleaning up the databases and profile between device tests.
• [rwood] Finished bluetooth discoverable test
• [rwood] Gaia-ui endurance tests:
  • Code updates after review by Dave
  • Setting up Jenkins now, will have more test results data for DataZilla soon

WebAPI (Rob)

• [rwood] Fix for WebTelephony test_outgoing_answer_hangup_oncallschanged test (bug 849714)
• [rwood] Fix WebTelephony intermittent failure test fallout by having all telephony tests delete any existing emulator calls at start-up (bug 811167)

WebRTC (Henrik)

• New data channel framework up for first feedback round on bug 796894
• Ted is working on the integration of faked devices

Softvision Updates

Desktop Automation

• Goals progress: Mozmill 2.0 release

• Fixed:
  • bug 864300 - EV certificate checks are failing for "https://addons.mozilla.org/licenses/5.txt"
  • bug 860382 - Test failure "Search engines drop down has been closed" in testSearchSuggestions.js
  • bug 834163 - Teardown code for /testSearch tests should not exercise ui path which can cause failures like: "Modal dialog has been found and processed"
  • bug 865211 - Test failure "aController.window.PlacesStarButton is undefined" in /testBookmarks_AddAndRemoveBookmarkViaAwesomeBar/test1.js
  • bug 671476 - Mozmill Endurance test for loading a SWF video via URL

• Blocked bugs:
  • bug 763159 - Test failure 'secure.mur.at == erle.mur.at ' in testSecurity/testUnknownIssuer.js
  • bug 795398 - Mozmill test to check that MD5 hash signatures are no longer accepted
    • needs CA certificate, Brandon answered to Henrik that it will be ready in a few days

• Working on:
  • Mozmill-ci - Log request/response bodies of update requests
    • trying with EnvInject plugin
  • bug 864375 - mozelement/elementslib should not invoke elements at declaration time
  • bug 732353 - Disable all Discovery Pane tests due to unpredictable web dependencies
  • bug 857551 - Create HTML page containing all the elements necessary for L10N testing of the context menu access keys
  • bug 671479 - Mozmill Endurance test for loading a SWF video via iframe
  • bug 763461 - Mozmill tests for cut-off elements should not report failure for scrollable direction
  • bug 763470 - False positive test for cut-off elements for advanced pane tabs
  • bug 764640 - Mozmill returns PASS/SKIP/FAIL == 0 for test_assertions.js
  • bug 860670 - mozmill.utils.getWindowByType fails for type: Addons:Blocklist - Fails with "Window has been found."
  • bug 864268 - Lookup class does no longer expose the `expression` property
  • Non-zero exit status 2.0 - Update automation script to re-raise exceptions in order to have a non-zero exit code

Mobile (Robocop)

• Progress on Robocop Automation
• Bugs with work in progress:
  • bug 820859 - Robocop: Add test for 'Clear Private Data'
  • bug 830834 - Robocop: Add test for 'Show Search Suggestions' feature
  • bug 846340 - Robocop: Add test for 'Clear Site Settings'
  • bug 836341 - Robocop: Add test for "Clear download history"
  • bug 838596 - Robocop: Add test for 'Master Password' feature
  • bug 846296 - Robocop: Add test for 'Add Search Engine' feature
  • bug 833832 - Robocop: Add test for Import from Android feature
  • bug 855978 - Robocop: upgrade to robotium-solo-4.0.jar
• Bugs recently integrated:
  • bug 864280 - Robocop: Cleanup of BaseTest.java.in
• Intermittent fails we investigated:
  • bug 836818 - Intermittent testOverscroll | Checking gecko didn't draw unnecessarily - got true, expected false
  • bug 851861 - Intermittent testOverscroll, testPanCorrectness, testAxisLocking, testLoad, testFlingCorrectness | Pixel at 100, 0 - Color rgba(0,0,0,255) not close enough to expected rgb(32,100,0)
  • bug 854043 - Intermittent testWebContentContextMenu | Verify that the download started - The download started
• Notes/Issues/Questions:
  • We have a few patches that pass on try but we are still seeing bug 663657. We have to wait until the issue is fixed before looking to integrate the patches

Personal Status

For the personal status please check the weekly status updates:

• 130429 - 130503

Meeting Notes

The meeting notes for this meeting can be read in the following etherpad:

• http://etherpad.mozilla.com:9000/automation-developer-meeting-20130429

Action items

• [NEW] Henrik: Send an email to the automation list about the name of the group being called automation
• [NEW] Henrik: File a bug to make public information visible (Mozillians)
• [NEW] Henrik: Push the mozmill 2.0 release after the meeting
• [NEW] Andrei: Send an email to Neil regarding bug 803492
• [CARRY OVER] Henrik: Update the goals page
• [CARRY OVER] Anthony: Ensure any Metro tests are suitably tagged with in-testsuite?