QA:Home Page:Firefox 3.0 TestPlan:Leaks:Topsites
Short Overview over the automated QA Global 500 TopSite Testrun
QA Contact: Carsten Book/Tomcat
QA is running now continuously a Global Topsite Tests on all platforms (Windows XP, Linux, Mac 10.5) and Vista using Firefox 1.9.1 Builds. In this Tests (currently) 500 Topsites like yahoo.com, google.com, google.de, cnn.com etc are used to test Firefox for Memory Leaks, Assertion and Crashs.
QA use for the Global Topsite run the Sisyphus Testframework (including the tester.sh script) and a url list of 500 Topsites.
The Current Tests visits the links 1 level deep for each site. Its possible to run tests for deeper levels, but this mean also that the testrun itself takes a lot more time. As example a Global 500 Test 1 Level Deep take around 5 days to complete.
With this Testrun we are also to log Memory Leaks and Assertions stack (using the fix-macos-stack.pl and fix-linux-stack.pl for Mac and Linux) into the Logfile.
Also we get a Testresult after every Site Test. As example:
http://www.<URL>.<here>: EXIT STATUS: CRASHED signal 5 SIGTRAP (181.386045 seconds)
The Global Topsite Test is a great Tool to Test Firefox on Topsites and to find Memory Leaks and Crashes/Assertion. The Topsite Tests is now also included for Firefox 3.1 Release Tests and for future Release Testing.
Also we plan to expand and improve this Topsite run in the future for Tests with extensions and additional configurations, especially gczeal and valgrind, to discover regressions/leaks.
For a List of Bugs found so far by using the Sisyphus Testframework, please see Bug 455273