Auto-tools/Projects/peptest: Difference between revisions

Jump to navigation Jump to search

Auto-tools/Projects/peptest (view source)

Revision as of 20:59, 3 October 2011

1,418 bytes added ,  3 October 2011
no edit summary
(Created page with "== Overview == Peptest is an automated testing framework designed to test whether or not the browser's UI thread remains responsive while performing a variety of actions. Tests...")
 
No edit summary
Line 9: Line 9:
=== Running Tests  ===
=== Running Tests  ===


Currently tests are run from the command line with python. Peptest currently depends on some external Mozilla python packages, namely: mozrunner, mozprocess, mozprofile and mozlog. These packages are in the process of being consolodated into one location but currently live in two locations.  
Currently tests are run from the command line with python. Peptest currently depends on some external Mozilla python packages, namely: mozrunner, mozprocess, mozprofile, mozlog and manifestdestiny. These packages are in the process of being consolidated into one location but currently live in three different locations.  


#Clone the mozmill and mozbase repositories at [https://github.com/mozautomation/mozmill github.com/mozautomation/mozmill] and [https://github.com/mozilla/mozbase github.com/mozilla/mozbase] respectively.  
#Clone the mozmill, mozbase and manifestdestiny repositories at [https://github.com/mozautomation/mozmill github.com/mozautomation/mozmill] , [https://github.com/mozilla/mozbase github.com/mozilla/mozbase] and [http://hg.mozilla.org/automation/ManifestDestiny/ hg.mozilla.org/automation/ManifestDestiny/] respectively. Alternatively many of these packages live on pypi and can be installed via the normal easy_install or pip method.  
#Create a new virtualenv and install the mozrunner, mozprofile, mozprocess and mozlog packages into it.  
#Create a new virtualenv and install the mozrunner, mozprofile, mozprocess, mozlog and manifestdestiny packages into it (run 'python setup.py install' for each).  
#Clone the repo at [https://github.com/ahal/peptest.git https://github.com/ahal/peptest.git]<br>  
#Clone the Peptest repo at [https://github.com/ahal/peptest.git https://github.com/ahal/peptest.git]<br>  
#Run the command <pre>python runpeptests.py -b  
#Run the command (use --help for a full list of commands)<br><pre>python runpeptests.py -b ''path_to_binary''--profile-path ''path_to_profile'' --test-path ''path_to_test_manifest''</pre>
 
=== Test Manifest  ===
 
All parameters are optional except for the --test-path parameter. This parameter should contain the path to a manifest that lists all the tests that will be run. Manifests should be of the following format:
<pre>#&nbsp;test paths are relative to the manifest
[test1.js]
[test2.js]
[foo/bar/test3.js]
</pre>
 
=== Test Format  ===
 
Tests are simply Javascript files that will be executed in chrome space. This means they have access to any of the API's that an extension would normally have access to. In addition to this, they also have access to [https://developer.mozilla.org/en/Mozmill developer.mozilla.org/en/Mozmill Mozmill]'s driver for convenient automation of Firefox/Thunderbird's UI.
 
It is helpful to think of tests as a series of actions. That is, the UI thread will only be checked for responsiveness while an action is currently happening. This ensures that we are only testing the actions that we care about, and that the test isn't overrun with noise generated during setup or teardown. To accomplish this, tests call a function called 'performAction(name, func)' which takes in two parameters, a name and a function pointer whose contents consist of a single action.
 
The following example test will make sure that the browser remains responsive while opening a page and while searching in Google.
<pre>
// import mozmill and initialize a controller
Components.utils.import('resource://mozmill/driver/mozmill.js');
let controller = getBrowserController();
 
// Create our first action which will open Google
performAction('open_google', function() {
  controller.open('http://google.com');
  controller.waitForPageLoad();
});
 
// stuff not inside a performAction() call won't be tested for responsiveness
let textbox = findElement.ID(controller.tabs.activeTab, 'lst-ib');
let button = findElement.Name(controller.tabs.activeTab, 'btnK');
 
// Create our second action which will perform a search in the google searchbox
performAction('search_google', function() {
  textbox.sendKeys('foobar');
  button.click();
  controller.waitForPageLoad();
});
</pre>
</pre>


=== Test Format === Tests are simple &lt;pre&gt;interface ProcessHandler { void __init__(string cmd, # Command to execute array args = None, # Array of arguments string cwd = None, # Current working directory MozEnv env = os.environ, # OS Environment ignore_children = False, # If false (default), track child processes logname = None, # Log file to write output to **kwargs) # Any other args to pass directly to Popen void run() # Runs the process, wait for finish must be called # TODO - can that ^ be automatic? void kill() # Kills underlying process and potentially all children /* Try to read a line of output from the file object |f|. |f| must be a pipe, like the |stdout| member of a subprocess.Popen object created with stdout=PIPE. If no output is received within |timeout| seconds, return a blank line. Returns a tuple (line, did_timeout), where |did_timeout| is True if the read timed out, and False otherwise. */ (string, bool) readWithTimeout(file f, # File int timeout) # Timeout in seconds # Called for each line output by the process - returns None void processOutputLine(line) # line from process writing to stdout/stderr # Called when process times out - returns None void onTimeout() # Called when process finishes without a timeout - returns None void onFinish() /** Handle process output until the process terminates or times out. If timeout is not None, the process will be allowed to continue for that number of seconds before being killed. If outputTimeout is not None, the process will be allowed to continue for that number of seconds without producing any output before being killed. If storeOutput=True, the output produced by the process will be saved as self.output. If logfile is not None, the output produced by the process will be appended to the given file. **/ int waitForFinish(int timeout = None, int outputTimeout = None, bool storeOutput = True, string logfile = None) Properties: bool timedOut # True if process timed out string output # Output of process bool isRunning # True if process still running, false otherwise
For documentation on using Mozmill's driver see: [https://developer.mozilla.org/en/Mozmill#Reference_Desk]
 
== Further Work ==
=== Mozmill e10s ===
 
Mozmill doesn't use SpecialPowers when it interacts with content. This means that it will not work correctly with Electrolysis builds (which is mostly the entire point of Peptests). Mozmill will need to be refactored to use SpecialPowers and also to stop using the gBrowser object which doesn't exist in mobile Firefox.
 
=== Detailed results ===
 
Dietrich has written some tools to determine exactly which Javascript calls were unresponsive. This is a very useful tool which should be incorporated into the Peptest harness. More details can be found here: [http://etherpad.mozilla.com:9000/responsiveness-profiling]
 
=== Buildbot Integration ===
 
Peptests will need to run on a per checkin basis. This means integrating with buildbot and all of the fun that that entails.
Ahal
Confirmed users
656

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/354258"

Navigation menu