TestEngineering/Web/Automation/AMO Hudson Logs Summary August 2010
Smoke tests amo.preview
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 9/1/10 | GREEN | ||||
| 8/31/10 |
1348 |
search personas by name |
see bug 590652 |
||
| 8/31/10 |
1344 - 1347 |
GREEN | |||
| 8/27/10 - 8/30/10 | 1325 - 1343 | search personas by name | search results page missing major elements | Failure due to changes in persona search results page. Test cases need to be updated. | |
| 8/26/10 |
12:01 pm |
1313 |
test cases with login |
user name input field not found on login screen |
login page on preview has changed |
| 8/215/10 - 8/26/10 | 1308 - 1312 | GREEN | |||
| 8/25/10 | 6:01 pm | 1307 | various test cases | erorr 500 - internal server error | |
| 8/24/10 - 8/25/10 | 1298 - 1306 | GREEN | |||
| 8/24/10 | 12:01 pm | 1297 | various test cases |
. error 500 - internal server error . timeout errors |
|
| 8/20/10 - 8/24/10 | 1261 - 1296 | GREEN | |||
| 8/20/10 | 12:01 mid | 1260 | various test cases | error 503 - service unavailable | |
| 8/19/10 | 1259 | GREEN | |||
| 8/19/10 | 6:04 pm | 1258 | various test cases |
. error 500 - internal server error . timeout errors . 'service unavailable' errors |
|
| 8/18/10 - 8/19/10 | 1251 - 1257 | personas landing layout |
. expected persona info not found in lists. . list headers and links to View All not as expected |
Top Rated list appears instead of Up & Coming (as column) on page again. Tests revised for new layout. | |
| 8/18/10 | 1239 - 1250 | GREEN | |||
| 8/17/10 | 1238 | error 500 - internal server error | |||
| 8/16/10 - 8/17/10 | 1234 - 1237 | GREEN | |||
| 8/16/10 | 3:29 pm | 1233 | all test cases | error 500 - internal server error | |
| 8/13/10 - 8/16/10 |
1209 - 1232 |
GREEN | |||
| 8/13/10 | 3:01 pm | 1208 | search add-ons by name |
. function not found in shared library . NO XHR error 500s |
timing issue when updating SVN. |
| 8/13/10 | 12:01 pm | 1207 | all test cases | XHR error 500 - Internal server error | |
| 8/12/10 - 8/13/10 | 1198 - 1206 | GREEN | |||
| 8/12/10 | 9:01 am | 1197 | addon category search's sorts | category missing in search panel. | same problem seen when suites experience 500's errors. No 500 errors in this run. AMO search suite at 8 am had 500's errors |
| 8/11/10 - 8/12/10 | 1194 - 1196 | GREEN | |||
| 8/11/10 | 6:01 pm | 1193 | persona search by name | persona not found in search results | |
| 8/10/10 - 8/11/10 | 1186 - 1192 | GREEN | |||
| 8/10/10 | 6:01 pm | 1185 | search add-on by name |
. Missing pagination element in listing footer . Add-on not found in search results. |
cause unknown. |
| 8/10/10 | 3:01 pm | 1184 | .XHR error 500 - Internal server error | ||
| 8/9/10 - 8/10/10 | 1178 - 1183 | GREEN | |||
| 8/9/10 | 1175 - 1177 | various test cases |
.XHR error 500 - Internal server error . Timeout waiting for page |
||
| 8/8/10 - 8/9/10 | 1168 - 1174 | GREEN | |||
| 8/8/10 | 12:00 noon | 1167 | various test cases | RC environment not available | |
| 8/7/10 - 8/8/10 | 1155 - 1166 | GREEN | |||
| 8/6/10 - 8/7/10 | 1142 - 1154 | . amo home page layout
. collections landing page layout |
.collections do not include author link | test cases are temporarily disabled pending fixed to web pages. | |
| 8/6/10 | 1142 |
amo home page layout | . collection on the page does not include author. . checking links for Popular Tags does not consider mixedcase tags. |
||
| 8/3/10 - 8/5/10 | based on running suite on a local machine - all test cases finish but job does not terminate. Job must be aborted manually. Job logs do no contain results from test cases. | investigation continues | |||
| 8/3/10 - 8/4/10 | 1121 - 1123 | GREEN | |||
| 8/3/10 | 1118 - 1120 | AMO home page layout | the View All Featured Add-ons link had an unexpected URL | Test case updated to check for updated URL. | |
| 8/3/10 | many | suite fails to start due to error with SVN | revolved with update to SVN url | ||
| 8/2/10 | 1086 - 1087 | GREEN | |||
| 8/1/10 |
no runs due to RC Grid issues |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset
Smoke tests amo.next
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 8/24/10 - 8/26/10 | 217 - 232 | GREEN | |||
| 8/24/10 |
216 |
suite |
suite failed to start. error 403 Forbidden |
||
| 8/22/10 - 8/24/10 |
198 - 215 |
GREEN | |||
| 8/22/10 |
197 |
add-on category landing page lists sorts |
error 500 - internal server error |
||
| 8/18/10 - 8/22/10 |
167 - 196 |
GREEN | |||
| 8/18/10 |
7:01 am |
166 |
various test cases |
Error 61 - Connection refused |
|
| 8/13/10 - 8/17/10 |
129 - 165 | GREEN | |||
| 8/13/10 | 4:01 pm | 128 | search add-ons by name |
. function not found in shared library . NO XHR error 500s |
timing issue when updating SVN. |
| 8/12/10 - 8/13/10 | 121 - 127 | GREEN | |||
| 8/12/10 | 4:01 pm | 120 | .XHR error 500 - Internal server error | ||
| 8/10/10 - 8/12/10 | 105 - 119 | GREEN | |||
| 8/10/10 | 4:01 pm | 104 | .XHR error 500 - Internal server error | ||
| 8/9/10 - 8/10/10 | 98 - 103 | GREEN | |||
| 8/9/10 | 7:01 pm | 97 | various test cases | . XHR error 500 - Internal server error | |
| 8/7/10 - 8/9/10 | 83 - 96 | GREEN | |||
| 8/7/10 | 82 | . AMO home page layout
. collections landing page layout |
author link is missing in collection listing | ||
| 8/3/10 - 8/5/10 | See other smoketests for same time period | ||||
| 8/3/10 | 75 | GREEN | |||
| 74 | amo home page layout | the View All Featured Add-ons link had an unexpected URL |
Test case updated to check for revised url | ||
| 8/3/10 | suite fails to start due to error with SVN | resolved with update to configures re: svn url | |||
| 8/2/10 - 8/1/10 |
test cases fail to start due to SVN issue |
Hudson configure updated to new SVN URL. |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset
Smoke tests amo.prod
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 8/31/10 | 30 - 31 | collection landing page layout | Conflict between TC for new collection pages and prod pages | Disabled test case for prod suite until new pages are in production | |
| 8/27/10 - 8/29/10 | 27 - 29 | GREEN | |||
| 8/27/10 | 26 | same errors as run #23 | |||
| 8/26/10 | 5:53 pm | 25 | same errors as run #23 | ||
| 8/26/10 | 5:29 pm | 24 | same errors as run #23 with different set of add-ons & collections | ||
| 8/26/10 | 4:45 pm | 23 | collections landing page layout | collection header missing author | Manual check showed author in header. |
| search add-on by name | headers different on browse page and search results | multiple add-ons. No difference seen when a few were manually checked. Difference might be caused by missing author. | |||
| 8/26/10 | 11:19 am | 22 | search persona by name | expected persona not found in search results. | /z/ appears intermittently in persona URLs on browse page. see bug 590652. |
| 8/26/10 | 21 | search add-on by name | description different on browse page and search results. |
add-on: HandyNote. No difference seen when manually checked. | |
| 8/25/10 |
20 |
search persona by name |
expected personas not found in search results. |
expanded # of personas checked in TC. Problem seen with personas listed on browse page with when Popular option is selected. Issue with URL on browse page. see bug 590652. | |
| 8/25/10 | 19 | search persona by name | expected persona not found in search results | persona: Sunset Over Water | |
| 8/24/10 | 18 | search add-on by name | expected add-on not found in search results | add-on: NoScript | |
| browse page and search results have different descriptions |
addon: Save-To-Read. did not seen a difference in debug mode. | ||||
| search persona by name | expected persona not found in search results | persona: "Sunset Over Water" | |||
| 8/23/10 | 17 | GREEN | |||
| 8/22/10 | 5:01 am | 16 | search persona by title | expected persona not found in search results | persona: "Sunset Over Water" |
| 8/20/10- 8/21/10 |
14 - 15 |
GREEN | |||
| 8/19/10 |
13 |
search add-on by name |
expected persona not found in search results |
personas: "AO-Maru Robot Mascot Racing TheGamerStop", "Sunset Over Water". Both found in manual search | |
| 8/18/10 | 12 | search persona by title | expected persona not found in search results | persona: "Sunset Over Water". Found in manual search | |
| 8/17/10 | 11 | search add-on by name | add-on description different on browse page and search results. | Did not appear in next run. No difference seen when visually checked. | |
| 8/14/10 - 8/16/10 |
8 - 10 | GREEN |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset
Other amo.preview
BFT
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 8/30/10 | 39 | test suite hang | aborted after running 10 hrs. (normal run 1.5 hrs.) | ||
| 8/28/10 - 8/29/10 | 36 - 38 | various test cases | major elements missing from pages | Failure in persona search due to changes in persona search results page. Test cases need to be updated. | |
| 8/26/10 - 8/27/10 |
34 - 35 | GREEN | |||
| 8/26/10 |
12:07 pm |
33 |
test cases with login |
user name input field not found on login screen |
login page on preview has changed |
| 8/25/10 - 8/26/10 |
31-32 |
persona landing page lists |
Top Rating lists not sorted in descending order |
||
| 8/24/10 | 30 | test src URL link | element not found | links in featured add-ons changes. test case updated for current featured add-ons and collections page. | |
| 8/21/10 - 8/23/10 |
26 - 28 |
add-on detail layout |
trivally different descriptions on browse and detail page |
bug 589231. test case updated to skip exception described in the bug. | |
| test src URL link | element not found | ||||
| 8/20/10 |
2:01 pm |
25 |
add-on detail layout |
trivally different descriptions on browse and detail page |
bug 589231 |
| search add-ons by app version | search failed. "Service Unavailable" error | ||||
| test src URL link | element not found | ||||
| 8/19/10 |
4:39 pm |
24 |
add-on detail layout |
1) add-on had different description on browse and detail page 2) website listed on detail page does not match normal pattern for URLs. |
1) addon: "307.to". differing descriptions need more investigation 2) addon: "tipish! uploader". link works fine. possibly revise TC and simply check for displayed URL since the link works. |
| test src URL link | element not found | ||||
| 8/19/10 | 2:01 pm |
23 | most test cases |
. error 500 - internal server error . timeout waiting for page |
|
| 8/19/10 |
4:33 pm |
22 |
various test cases |
. error 500 - internal server error |
|
| addon detail layout |
. add-on had different description on browse and detail page . missing element in a header |
. differing descriptions visually appear identical. need to investigate further with debugger. . missing element may be related to 500 errors seen later. | |||
| test src URL link |
element not found |
||||
| 8/18/10 | 2:01 pm | 20 | various collections test caes | elements not found | collections test cases disabled until collections pages are stable and test cases are updated. |
| test src URL link | element not found | needs futher investigation | |||
| 8/18/10 | 12:25 pm |
19 | add-on browse page sorts | add-ons without authors thrown as exceptions | need to update TC to verify author-less add-ons are rare |
| various test cases | page missing elements | need to investigate. no 500 errors seen. | |||
| 8/13/10 - 8/15/10 |
13 - 15 |
. XHR error 500 - Internal server error. . Missing page elements . Unexpected URL for collections . Search for collections does not return expected collection. |
. Investigate if expected collection URL has changed . Investigate collection search further. | ||
| 8/13/10 | 2:01 pm | 12 | .XHR error 500 - Internal server error | ||
| 8/12/10 |
2:01 pm | 11 |
. XHR error 500 - Internal server error . timeout wating for page. . missing page elements. |
||
| 8/10/10 - 8/11/10 | 2:01 pm | 9 - 10 | various test cases |
. listings of collections are missing author link . add-on appears without a title . XHR error 500 - Interal server error |
. missing author link is expected to be resolved in the near future. . add-on without title needs further investigation |
| 8/9/10 | 8 | various test cases |
.XHR error 500 - Internal server error .Timeout waiting for page |
||
| 8/7/10 - 8/8/10 | 6 - 7 | various test cases | listings of collections are missing author link | ||
| 8/7/10 - 8/8/10 | 6 - 7 | collection landing layout | special user with 1 collection not setup as expected. | ||
| 8/3/10 - 8/5/10 |
see comments on Smoketest for same time period |
||||
| 7/31/10 - 8/2/10 |
4 |
run #4 hung. not finished after weekend. |
disabled while problem is investigated. |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset
FFT
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 8/31/10 | 103 | GREEN | |||
| 8/28/10 - 8/29/10 | 101 - 102 | various test cases | major element missing from pages | Failures in persona searches due to changes in persona search results page. Test cases need to be updated. | |
| category landing page layout | Add-ons in Mobile previous had 2 green buttons for installs, now 1 button. | Web pages changed. Scripts need to be updated. | |||
| 8/26/10 - 8/27/10 |
99 - 100 |
GREEN | |||
| 8/25/10 | 11:01 am | 98 | addon category landing page layout | login name box not found in 1 of the 5 iterations. | not seen in manual check |
| addon category landing page list sorts | part of page header missing in 2 of the 5 iterations. | not seen in manual check | |||
| 8/24/10 |
11:01 am |
97 |
persona search by title |
unrecognized page returned |
persona: Pinguinos Madagascar |
| "created" and "popular" sort options missing from page | category: "Film and TV" | ||||
| 8/18/10 - 8/23/10 |
91 - 96 |
GREEN | |||
| 8/16/10- 8/17/10 |
89 - 90 | add-on category landing page layout | text on Install button does not match expected patterns |
. Revise verifyInstallButton( ) to include 'Download for OS' buttons. . Investigate why null is returned for Install button text when visually the buttons have text. | |
| add-on category landing page layout and lists | see runs # 66 - 74 | ||||
| 8/14/10 - 8/16/10 | 86 - 88 | add-on category search's sort | no search results. | possible 500 error? | |
| search panel category #2 is missing | |||||
| add-on category landing page layout | Python error | probable coding error in test cases. error occurs and a recently modified function. | |||
| add-on category landing page lists | see runs #66 - 74 | ||||
| 8/13/10 | 11:01 am | 85 | all tests | error 500 - Internal server error | |
| 8/12/10 | 84 | add-on category landing page layout | see runs #66 - 74 | known issue | |
| various test caess | elements missing from page | similar to issues seen in other suite runs with 500's errors | |||
| 8/11/10 | 11:16 am | 83 | various test cases | elements not found | |
| 8/10/10 | 82 | add-on category landing page layout | see runs #66 - 74 | known issue | |
| 8/9/10 | 81 | persona search by title | . elements missing on browse page
. browse page returned no personas . search returned no results for a known persona |
||
| 8/7/10 - 8/9/10 | 78 - 81 | add-on category landing page lists | see runs #66 - 74 | known issue | |
| 8/3/10 - 8/4/10 | FFT suite | see comment on smoketest hangs | |||
| 8/2/10 | 76 | run not completed after 2.5 hrs. run aborted | suite disabled while problem is investigated. | ||
| 8/1/10 |
no runs due to BFT hang. |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset
Search subset
| Date | Time | Run | Test Case | Issue | Resolution / Comment |
|---|---|---|---|---|---|
| 8/31/10 | 99 | GREEN | |||
| 8/28/10 - 8/31/10 | 96 - 98 |
persona search by name | search results pages missing major element |
Failure due to changes in persona search results page. Test cases need to be updated. Update: Updating test cases resolved some issues. Remaining failure handled by skipping persona categories in test run until persona pages are stable again. | |
| 8/26/10 | 93 | GREEN | |||
| 8/25/10 | 5:26 pm | 92 | various |
. error 500 - internal server error . elements missing on pages |
|
| 8/14/10 - 8/25/10 |
78 - 91 | search by name | two add-ons not found in search results. |
Repeatable. 1) "Provider for Google Calendar" in Sunbird 2) "Lightning" in SeaMonkey === Update: Although listed in Sunbird & SeaMonkey browse page the add-on page displays them as part of Thunderbird. Bug 588215 Update 2: test case updated to suppress exception for these combinations of app & addon | |
| search on collections | see run # 77 | Not seen in manual search or TC run on local machine | |||
| 8/13/10 |
7:39 pm |
77 |
search on collections |
searches failed or did not return expected collection in results |
Not seen in manual search. |
| addon search by name |
search for "Provider for Google Calendar" in Sunbird does not return the add-on |
Repeatable. Needs to be investigated further | |||
| 8/13/10 | 10:26 am | 76 | all test cases | .XHR error 500 - Internal server error | |
| 8/13/10 | 8:01 am | 75 | search collections by name | see runs #49 - 53 | known exceptions |
| search add-ons by name | expected Next Page link is missing | ||||
| 8/12/10 |
8:01 am |
74 |
search collections by name |
see runs #49 - 53 |
known exceptions |
| various test cases |
.XHR error 500 - Internal server error . various page elements missing |
||||
| 8/11/10 |
8:01 am, 10:48 am |
71, 72 |
collections search by name |
see runs #49 - 53 |
known exceptions |
| various test cases |
. XHR error 500 - Internal server error . various page elements missing |
||||
| 8/10/10 | 70 | collections search by name | see runs #49 - 53 | known exceptions | |
| 8/9/10 | 12:51 pm | 69 | various test cases | XHR error 500 - Internal server error | |
| 8/7/10 - 8/9/10 | 66 - 68 | suite fails | attempting to access retired test case | result of reverting to a previous suite | |
| 8/3/10 - 8/5/10 | search suite | see comments on smoketest hangs for same time period | |||
| 8/2/10 | 65 | run not finished after 2.5 hours. run aborted | large suites disabled while problem is investigated. | ||
| 8/1/10 |
no runs due to BFT hang. |
go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset