TestEngineering/Web/Automation/AMO Hudson Logs Summary

< TestEngineering‎ | Web‎ | Automation
Revision as of 17:31, 20 August 2010 by Twsmith (talk | contribs)

Smoke tests amo.preview


Date Time Run Test Case Issue Resolution / Comment
8/20/10 1261 - 1263 GREEN
8/20/10 12:01 mid 1260 various test cases error 503 - service unavailable
8/19/10 1259 GREEN
8/18/10 - 8/19/10
1251 - 1256 personas landing layout

. expected persona info not found in lists.

. list headers and links to View All not as expected

Top Rated list appears instead of Up & Coming (as column) on page again.  Tests revised for new layout.
8/18/10
1239 - 1250

GREEN
8/17/10
1238
error 500 - internal server error
8/16/10 - 8/17/10
1234 - 1237

GREEN
8/16/10 3:29 pm 1233 all test cases error 500 - internal server error
8/13/10 - 8/16/10

1209 - 1232


GREEN
8/13/10 3:01 pm 1208 search add-ons by name

. function not found in shared library

. NO XHR error 500s

timing issue when updating SVN.
8/13/10 12:01 pm 1207 all test cases XHR error 500 - Internal server error
8/12/10 - 8/13/10
1198 - 1206

GREEN
8/12/10 9:01 am 1197 addon category search's sorts category missing in search panel. same problem seen when suites experience 500's errors.  No 500 errors in this run. AMO search suite at 8 am had 500's errors
8/11/10 - 8/12/10
1194 - 1196

GREEN
8/11/10 6:01 pm 1193 persona search by name persona not found in search results
8/10/10 - 8/11/10
1186 - 1192

GREEN
8/10/10 6:01 pm 1185 search add-on by name

. Missing pagination element in listing footer

. Add-on not found in search results.

cause unknown. 
8/10/10 3:01 pm 1184
.XHR error 500 - Internal server error
8/9/10 - 8/10/10
1178 - 1183

GREEN
8/9/10
1175 - 1177 various test cases

.XHR error 500 - Internal server error

. Timeout waiting for page


8/8/10 - 8/9/10
1168 - 1174

GREEN
8/8/10 12:00 noon 1167 various test cases RC environment not available
8/7/10 - 8/8/10
1155 - 1166

GREEN
8/6/10 - 8/7/10
1142 - 1154 . amo home page layout

. collections landing page layout

.collections do not include author link test cases are temporarily disabled pending fixed to web pages.
8/6/10
1142
amo home page layout . collection on the page does not include author.

. checking links for Popular Tags does not consider mixedcase tags.


8/3/10 - 8/5/10


based on running suite on a local machine - all test cases finish but job does not terminate.  Job must be aborted manually.  Job logs do no contain results from test cases. investigation continues
8/3/10 - 8/4/10
1121 - 1123

GREEN
8/3/10
1118 - 1120 AMO home page layout the View All Featured Add-ons link had an unexpected URL Test case updated to check for updated URL.
8/3/10
many
suite fails to start due to error with SVN revolved with update to SVN url
8/2/10
1086 - 1087

GREEN
7/31/10 - 8/1/10



no runs due to RC Grid issues
7/29/10 - 7/30/10
1077 - 1084

GREEN
7/29/10 1:56 pm 1076 add-on category search sorts exception in name sort same TC problems with sort comparisons in other suites.  assuming this is related.  TC handling by suited driver is being investigated.

12:00 noon 1075 multiple test cases . XHR error 500 - Internal server error

. Timeout error on page opens




1074 - 1071

GREEN
7/28/10 9:44 pm 1070 multiple test cases . XHR error 503 - Serice unavailable

. searches returned no results . element not found




1069

GREEN

6:00 pm 1068 multiple test cases XHR error 500 - Internal server error


1066, 1067

GREEN
7/28/10 3:28 pm 1065 multiple test cases XHR error 500 - Internal server error


1064

GREEN


1063 add-on category search sorts Exception in sort by download count (2), sort by name (1)


1062 persona landing page layout various elements missing on pages


1060 - 1061

runs aborted


1059 multiple test cases XHR error 500 - Internal server error


1058

GREEN


1057 add-on category search sorts Python code failure different from run #1055 no code changes immediately before or after
7/28/10

1056

GREEN
7/27/10

1055 add-on category search sorts Python code failure no code changes immedilately before or after


1054

GREEN
7/27/10

1051
Add to Firefox install buttons
Could not find RC environment
TC corrected


1051- 1053
add-on category search sorts
Exceptions re: sorting add-ons by rating
Test w/ 'rating sort' parameter which was previously disabled became enabled in error.  Corrected after run # 1053
7/27/10 6:29 pm 1050 various test case Could not find RC enviroments Appeared to be grid errors
7/26/10 12:00 noon 1032 two tests failed XHR error 500 - Internal server error Green starting run #1033
7/25/10 6:00 pm 1022 most tests had failures . Searches failed

. element not present on the page

Green starting run #1023
7/22/10
997 - 998 AMO home page layout exception on expected URL pattern for Contribute element
URL pattern updated.  Resolved as of run 999


996 - 991
AMO home page layout
collections' URL contains /anonymous/.  links are broken
not seen in run #997


964 - 966
add-on category search sorts
search failed on 'all categories' or individual categories




addon search by name
search failed. 
No useful info returned due to code error in search fcn.


962 - 963
add-on search by name
search returned no results for "1 Click Bookmark"

7/19/10
9:00 am
961
test search API mobile
Error 503 - Service unavailable




add-on category search sorts . no add-ons verified for Name sort
. no add-ons verified for Newest sort
Needs to be researched


958 - 960
add-on search by name
failed search.
Due to coding error in search fcn no useful info was returned.

12:00 mid 958 persona search by title 'Most Popular' sort button not found




no search results returned for "DrayTek normal Logo"

7/18/10

* Logs not summarized *

7/17/10


* Logs not summarized * 


7/16/10

932 - 939
home page layout
see run 916 re: missing elements
Error no longer reported as of run #940, 7/16 @ 6 pm
7/16/10
12:00 mid
931
home page layout
see run 916 re: missing elements




persona search by title
- 'Most Popular' option not found on personas browse page
- search returned no results for 'DrayTek normal Logo'

7/15/10

929 - 930
home page layout
see run 916 re: missing elements


3:01 pm
928
home page layout
see run 916 re: missing elements




persona search by title
- 'Recently Added' option not found on personas browse page
- search returned no results for "Magic Island"



926 - 927
all
Error 500 - Internal server error


11:08 am
925
addon search by name
search returned no results for "1 Click Bookmark"


9:39 am
924
test search API mobile
Error 503 - Service Unavailable



918 - 923
all
RC environment issues



917; 921+ home page layout see run 916 re: missing elements

7/14/10
5:16 pm
916
test search API platform
Error 503 - Service Unavailable




home page layout
elements missing from featured add-ons panel
bug in preview.amo
7/14/10 12:01 noon 911 all "Unknown environment" error Caused by grid/RC changes.  Resolved.



persona search by title

- 'Recently Added' option not found on personas browse page

- search returned no results for "Rocas y mar"



3:00 am
908
test search API platform
Error 503 - Service Unavailable




test search API type
Error 503 - Service Unavailable

7/13/10
9:00 pm
906
persona search by title
- 'Recently Added' option not found on personas browse page
- search returned no results for "Rocas y mar"


3:00 pm 904 add-on category search sorts - search returned no results for all-categories
- search returned no results for category Alerts & Updates


2:05 pm
903
persona search by title
- 'Recently Added' option not found on persona browse page
- search returned no results for "circle M&M"


12:00 noon
901
test search API type
Error 503 - Service Unavailable




addon search by name
search returned no results for "1 Click Bookmark"


9:00 am
900
test search API version
Error 503 - Service Unavailable


12:00 mid
897
test search API apps

Socket error - errno 8





test search API cat
- Socket error - errno 8

- No registered session...





test search API limit
- Socket error - errno 8

- No registered session...





test search API mobile
- Socket error - errno 8

- No registered session...





test search API type
Error 503 - Service Unavailable


go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset

Smoke tests amo.next


Date Time Run Test Case Issue Resolution / Comment
8/18/10 - 8/20/10
167 - 183
GREEN
8/18/10
7:01 am
166
various test cases
Error 61 - Connection refused


8/13/10 - 8/17/10

129 - 165

GREEN
8/13/10 4:01 pm 128 search add-ons by name

. function not found in shared library

. NO XHR error 500s

timing issue when updating SVN.
8/12/10 - 8/13/10
121 - 127

GREEN
8/12/10 4:01 pm 120
.XHR error 500 - Internal server error
8/10/10 - 8/12/10
105 - 119

GREEN
8/10/10 4:01 pm 104
.XHR error 500 - Internal server error
8/9/10 - 8/10/10
98 - 103

GREEN
8/9/10 7:01 pm 97 various test cases . XHR error 500 - Internal server error
8/7/10 - 8/9/10
83 - 96

GREEN
8/7/10
82 . AMO home page layout

. collections landing page layout

author link is missing in collection listing
8/3/10 - 8/5/10


See other smoketests for same time period
8/3/10
75

GREEN


74 amo home page layout the View All Featured Add-ons link had an unexpected URL
Test case updated to check for revised url
8/3/10


suite fails to start due to error with SVN resolved with update to configures re: svn url
8/2/10 - 7/31/10



test cases fail to start due to SVN issue
Hudson configure updated to new SVN URL.
7/30/10
63, 66

GREEN
7/29/10
59 - 62 amo home page layout collections links are broken. issue seen previously and disappeared without explanation.
7/29/10

59, 62
add-on category search sort
see run #53
debug statements indicate that a variable is getting unexpected values.  one theory is the variable is shared with other concurrent instances of the same TC run from the new suite driver.


58 add-on category search sort see run #53 Temporarily enabled TC verbosity and added debug print.
7/28/10
56 - 57 personas category landing page layout HTTP eror 500 - Internal server error.
7/28/10 7:00 pm 55 add-on category search sorts Exceptions with add-on sorts by rating Disabled TC with 'rating sort' parameter which was enabled in error.  Disabled agains after run # 55
7/28/10 4:00 pm 54

GREEN
7/28/10 1:00 pm 53 add-on category search sorts Python code error no code changes immediately before or after
7/27/10 6:28 pm 52 all tests Error 61 - Connection refused
7/25/10 7:00 pm 31 most test cases had failures similar to run #30 Green starting run #32
7/25/10
4:00 pm
30
most test cases had failures

. Searchs failed

. XHR error 503 - Service unavailable

. element not found on page



go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset

Smoke tests amo.prod


Date Time Run Test Case Issue Resolution / Comment
8/20/10 14 GREEN
8/19/10

13
search add-on by name
expected persona not found in search results
personas: "AO-Maru Robot Mascot Racing TheGamerStop", "Sunset Over Water".  Both found in manual search
8/18/10
12 search persona by title expected persona not found in search results persona: "Sunset Over Water".  Found in manual search
8/17/10
11 search add-on by name add-on description different on browse page and search results. Did not appear in next run.  No difference seen when visually checked.
8/14/10 - 8/16/10

8 - 10

GREEN


go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset

Other amo.preview

BFT

Date Time Run Test Case Issue Resolution / Comment
8/19/10
4:39 pm
24
addon detail layout

1) add-on had different description on browse and detail page

2) website listed on detail page does not match normal pattern for URLs.

1) addon: "307.to".  differing descriptions need more investigation

2) addon: "tipish! uploader". link works fine.  possibly revise TC and simply check for displayed URL since the link works.

test src URL link element not found
8/19/10 2:01 pm
23 most test cases

. error 500 - internal server error

. timeout waiting for page


8/19/10
4:33 pm
22
various test cases
. error 500 - internal server error




addon detail layout

. add-on had different description on browse and detail page

. missing element in a header

. differing descriptions visually appear identical.  need to investigate further with debugger.

. missing element may be related to 500 errors seen later.




test src URL link
element not found

8/18/10 2:01 pm 20 various collections test caes elements not found collections test cases disabled until collections pages are stable and test cases are updated.



test src URL link element not found needs futher investigation
8/18/10 12:25 pm
19 add-on browse page sorts add-ons without authors thrown as exceptions need to update TC to verify author-less add-ons are rare



various test cases page missing elements need to investigate.  no 500 errors seen.
8/13/10 - 8/15/10

13 - 15

. XHR error 500 - Internal server error.

. Missing page elements

. Unexpected URL for collections

. Search for collections does not return expected collection.

. Investigate if expected collection URL has changed

. Investigate collection search further.

8/13/10 2:01 pm 12
.XHR error 500 - Internal server error
8/12/10
2:01 pm 11

. XHR error 500 - Internal server error

. timeout wating for page.

. missing page elements.


8/10/10 - 8/11/10 2:01 pm 9 -  10 various test cases

. listings of collections are missing author link

. add-on appears without a title

. XHR error 500 - Interal server error

. missing author link is expected to be resolved in the near future.

. add-on without title needs further investigation

8/9/10
8 various test cases

.XHR error 500 - Internal server error

.Timeout waiting for page


8/7/10 - 8/8/10
6 - 7 various test cases listings of collections are missing author link
8/7/10 - 8/8/10
6 - 7 collection landing layout special user with 1 collection not setup as expected.
8/3/10 - 8/5/10



see comments on Smoketest for same time period

7/31/10 - 8/2/10

4

run #4 hung.  not finished after weekend.
disabled while problem is investigated.
7/29/10
2:06 pm
3
collection search by name
see runs # 25 - 32
known exceptions



add review
element not found error




check link and url src browser parameter error
7/28/10 8:35 pm
2 collection search by name see runs # 25 - 32 known exceptions



add review element not found error



persona search by title element not found error capture failure in an exception



check link and url src element not found error
7/28/10
1 various test cases XHR error 500 - Internal server error



add-on search by version element on advanced search panel not found



check links and src values browser parameter error





run numbers reset due since amo.bft running in a new queue
7/26/10 2:00 pm 33 various test cases various server errors
7/19/10

25 - 32
search collections by name
differences in data displayed on browse page and search results
collections pages will be updated in the near future.  will review remaining exceptions for action on new pages.
7/19/10
6:25 pm
25
search collections by name
differences in data displayed on browse page and search results
under review.  some may be considered non-exceptions.  others may be reports but not generate exceptions



search collections by name
missing elements under 'My Favorites' and 'My Collections'
Collections are not always listed under the selections.  TC needs to be fixed.



test src URL link
element not found
TC needs to be fixed
7/15/10
6:20 pm
23
search add-ons by version
element not found
to be investigated



search add-ons by version . no results returned for "3.7"
. no results returned for "3.6"




search personas using title
element not found
to be investigated



home page layout
element not found
known bug



add-on search by name
search returned no results for "NoScript"




collections landing layout
test user login failed.
possible issue with special test users used for TC.



collection search by name
search returned no results for "Web Developer's Toolbox"

7/15/10
1:31 pm 22 all Error 500 - Internal server error
7/14/10
1:59 pm
21
test src URL link
element not found
getting resolved



most test cases
login failed - acct name not found
login fcn was later changed to not check for acct name; check for 'log out' link remains.








go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset

FFT

Date Time Run Test Case Issue Resolution / Comment
8/18/10 91 - 92
GREEN
8/16/10- 8/17/10
89 - 90 add-on category landing page layout text on Install button does not match expected patterns

. Revise verifyInstallButton( ) to include 'Download for OS' buttons.

. Investigate why null is returned for Install button text when visually the buttons have text.

add-on category landing page layout and lists see runs # 66 - 74
8/14/10 - 8/16/10
86 - 88 add-on category search's sort no search results. possible 500 error?




search panel category #2 is missing



add-on category landing page layout Python error probable coding error in test cases.  error occurs and a recently modified function.



add-on category landing page lists see runs #66 - 74
8/13/10 11:01 am 85 all tests error 500 - Internal server error
8/12/10
84 add-on category landing page layout see runs #66 - 74 known issue



various test caess elements missing from page similar to issues seen in other suite runs with 500's errors
8/11/10 11:16 am 83 various test cases elements not found
8/10/10
82 add-on category landing page layout see runs #66 - 74 known issue
8/9/10
81 persona search by title . elements missing on browse page

. browse page returned no personas

. search returned no results for a known persona


8/7/10 - 8/9/10
78 - 81 add-on category landing page lists see runs #66 - 74 known issue
8/3/10 - 8/4/10

FFT suite see comment on smoketest hangs
8/2/10
76
run not completed after 2.5 hrs.  run aborted suite disabled while problem is investigated.
7/31/10 - 8/1/10



no runs due to BFT hang.

7/20/10 - 7/29/10
66 - 74
add-on category landing page lists
Landing page lists are empty. app: Mobile; category: Sports & Games
known bug.  Category count indicates 1 add-on in category.
7/29/10 11:00 am
74 add-on category search sorts
exceptions on sort by rating
previously suppression was undone.



search for personas by title
XHR error 500 - Internal server error




search for personas by title browser parameter error should be resolved with Hudson patch applied in the afternoon
7/20/10 - 7/23/10

66 - 68
add-on category landing page lists
Items in 'Top Rated' list are not sorted
Checking the sort is disabled until a new verification method is decided
7/14/10

64
category landing lists
element not found in Top Rated list
Possibly due to a not rated add-on.  TC needs to be enhanced and/or locator corrected




Top Rated list - add-ons not sorted.  See Thunderbird app.



persona search
element not found
needs research




search returned no results for "Magic Island"




category landing layout
install button not visible
add-on with multiple install links based on platform.  TC needs to be enhanced.




Top Rated list - rating element not always found
Possibly due to not rated add-ons.  TC needs to be enchanced and/or locator corrected




app Mobile, category 'Sport & Games' missing add-ons


go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset

Search subset

Date Time Run Test Case Issue Resolution / Comment
8/14/10 - 8/19/10

78 - 85 search by name two add-ons not found in search results.

Repeatable.

1) "Provider for Google Calendar" in Sunbird

2) "Lightning" in SeaMonkey

===

Update: Although listed in Sunbird & SeaMonkey browse page the add-on page displays them as part of Thunderbird.  Bug 588215




search on collections see run # 77 Not seen in manual search or TC run on local machine
8/13/10
7:39 pm
77
search on collections
searches failed or did not return expected collection in results
Not seen in manual search.



addon search by name
search for "Provider for Google Calendar" in Sunbird does not return the add-on
Repeatable.  Needs to be investigated further
8/13/10 10:26 am 76 all test cases .XHR error 500 - Internal server error
8/13/10 8:01 am 75 search collections by name see runs #49 - 53 known exceptions



search add-ons by name expected Next Page link is missing
8/12/10
8:01 am
74
search collections by name
see runs #49 - 53
known exceptions



various test cases

.XHR error 500 - Internal server error

. various page elements missing


8/11/10
8:01 am, 10:48 am
71, 72
collections search by name
see runs #49 - 53
known exceptions



various test cases

. XHR error 500 - Internal server error

. various page elements missing


8/10/10
70 collections search by name see runs #49 - 53 known exceptions
8/9/10 12:51 pm 69 various test cases XHR error 500 - Internal server error
8/7/10 - 8/9/10
66 - 68 suite fails attempting to access retired test case result of reverting to a previous suite
8/3/10 - 8/5/10

search suite see comments on smoketest hangs for same time period
8/2/10
65
run not finished after 2.5 hours.  run aborted large suites disabled while problem is investigated.
7/31/10 - 8/1/10



no runs due to BFT hang.

7/29/10
8:00 am
62
collections search by name
see runs #49 - 53
known exceptions



personas search by title browser parameter error should be fixed in the future by Hudson patches
7/28/10 12:41 pm 61 collections search by name see runs # 49 - 53 know exceptions



add-on search by add-on type see run #57 TC is obsolete because option was removed from search panel


59 - 60

run aborted
7/28/10 8:00 am 58 most test cases XHR error 500 - Internal server error



search personas by title Error with browser parameter
7/27/10 7:36 pm 57 add-on search by add-on type element not found possible coding error as it's a new script.  to be investigated



search personas by title Error with browser parameter



search collections by name see runs #49 - 53


54 - 56
runs maually aborted due to congestion
7/21/10 - 7/27/10

49 - 53
search collections by name
differences in data displayed on browse page and search results
collection pages will be updated in the near future.  waiting to see updates.
7/21/10
9:49 am
48
add-on search by name
Landing page lists are empty. app: Mobile; cateogry: Sports & Games




search collections by name
differences in data displayed on browse page and search results
under review. some may be considered non-exceptions. others may be reports but not generate exceptions
7/20/10
1:07 pm
47
search collections by name
differences in data displayed on browse page and search results
under review. some may be considered non-exceptions. others may be reports but not generate exceptions



add-on search by name
Landing page lists are empty. app: Mobile; cateogry: Sports & Games




addon search by version
Bug in TC in referencing locator data




persona search by title
Errno 61 - Connection refused


go to Smoke tests preview | next | production Other amo.preview BFT | FFT | Search subset