Web Testing/Automation/automation-27-05-2011: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(8 intermediate revisions by 3 users not shown)
Line 2: Line 2:
== Discussion topics ==
== Discussion topics ==


* Code Review Process
* [https://wiki.mozilla.org/Web_Testing/Automation/CodeReviewProcess Code Review Process]
** We need a policy on how long should one wait for his code to get merged.
** We need a policy on how long should one wait for his code to get merged.
*** Vishal suggests: 2 hours for failed test corrections and 2 days for new tests
*** Vishal suggests: 2 hours for failed test corrections and 2 days for new tests
*** We should have more people willing to review/merge code. We can't always rely on David and Dave to review/merge since they have other things to do.
*** We should have more people willing to review/merge code. We can't always rely on David and Dave to review/merge since they have other things to do.
*** If everybody thinks that the other person might/should review/merge then it never gets done.
*** If everybody thinks that the other person might/should review/merge then it never gets done.
* Our Firefox 4.01 builds are crashing, here: https://bugzilla.mozilla.org/show_bug.cgi?id=624800 (but only for certain tests+combinations, iirc, but we should double-check).
* Our Firefox 4.01 builds are crashing, here: https://bugzilla.mozilla.org/show_bug.cgi?id=624800
** http://screencast.com/t/0b26p91jRFm - check out 3:17 -- amo.rewrite; the empty-search testcase
** This is causing a lot of _spurious_ AMO test failures -- the failure ends up being reported in the exception; where the failure is reported is, in most cases, just where the browser crashed
** This is causing a lot of _spurious_ AMO test failures -- the failure ends up being reported in the exception; where the failure is reported is, in most cases, just where the browser crashed
*** Have we tried disabling crash reporter?  Do we want to?  On one hand, we're helping test the browser, but are also getting hit by it.
*** Have we tried disabling crash reporter?  Do we want to?  On one hand, we're helping test the browser, but are also getting hit by it.
****How about side-by-side runs (Firefox 4.01 vs. Firefox Beta (or Aurora))?
****How about side-by-side runs (Firefox 4.01 vs. Firefox Beta (or Aurora))?
**'''I've disabled crons for most AMO tests, for the time-being -- they're on-demand only, until we can figure out the failures'''
*AMO.search failure seems real; CSS locators need updating?  http://qa-selenium.mv.mozilla.com:8080/job/amo.search/lastBuild/console
* FYI, the MDN tests have been forked to the Mozilla Github account. The Jenkins builds have been moved too!


==Status Update==
==Status Update==
Line 19: Line 23:


===[https://wiki.mozilla.org/Sumo SUMO] [http://qa-selenium.mv.mozilla.com:8080/view/SUMO%20Tests/ CI]===
===[https://wiki.mozilla.org/Sumo SUMO] [http://qa-selenium.mv.mozilla.com:8080/view/SUMO%20Tests/ CI]===
*  
* Added 4 test methods for [https://github.com/mozilla/sumo-tests/pull/7 Knowledge Base articles]


=== [https://wiki.mozilla.org/Firefox/Input Input] [http://qa-selenium.mv.mozilla.com:8080/job/input%20stage/ CI] ===
=== [https://wiki.mozilla.org/Firefox/Input Input] [http://qa-selenium.mv.mozilla.com:8080/job/input%20stage/ CI] ===
Line 31: Line 35:


===MDN [http://qa-selenium.mv.mozilla.com:8080/view/MDN/ CI]===
===MDN [http://qa-selenium.mv.mozilla.com:8080/view/MDN/ CI]===
*No update
* CI and Github repo moved


===Mozilla.com [http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com/ CI]===
===Mozilla.com [http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com/ CI]===

Latest revision as of 18:05, 27 May 2011

<< Last meeting | WebQA Automation | Next meeting >>

Discussion topics

  • Code Review Process
    • We need a policy on how long should one wait for his code to get merged.
      • Vishal suggests: 2 hours for failed test corrections and 2 days for new tests
      • We should have more people willing to review/merge code. We can't always rely on David and Dave to review/merge since they have other things to do.
      • If everybody thinks that the other person might/should review/merge then it never gets done.
  • Our Firefox 4.01 builds are crashing, here: https://bugzilla.mozilla.org/show_bug.cgi?id=624800
    • http://screencast.com/t/0b26p91jRFm - check out 3:17 -- amo.rewrite; the empty-search testcase
    • This is causing a lot of _spurious_ AMO test failures -- the failure ends up being reported in the exception; where the failure is reported is, in most cases, just where the browser crashed
      • Have we tried disabling crash reporter? Do we want to? On one hand, we're helping test the browser, but are also getting hit by it.
        • How about side-by-side runs (Firefox 4.01 vs. Firefox Beta (or Aurora))?
    • I've disabled crons for most AMO tests, for the time-being -- they're on-demand only, until we can figure out the failures
  • AMO.search failure seems real; CSS locators need updating? http://qa-selenium.mv.mozilla.com:8080/job/amo.search/lastBuild/console
  • FYI, the MDN tests have been forked to the Mozilla Github account. The Jenkins builds have been moved too!

Status Update

AMO | CI

  • Test Day was a HUGE success!!!!

SUMO CI

Input CI


Socorro CI


MDN CI

  • CI and Github repo moved

Mozilla.com CI

  • No update

Action Items

Retrieved from "https://wiki.mozilla.org/index.php?title=Web_Testing/Automation/automation-27-05-2011&oldid=313464"