Litmus:Web Services

Litmus Web Services Specification

Sending testcases to the client

TBA

Sending results to the server

Submission

Testing clients will open an http connection to 'http://litmus.mozilla.org/process_test.cgi'. Clients must send via HTTP POST an XML document containing test results and metadata for submission. Results are formatted as described in the Test Result Format DTD. The parameter name will be 'data'.

Upon receipt, the server will process the test results. If a fatal error occurs, then no results will be added to the database. If some results are formatted properly, those results will be added even though other results may have errors.

The Test::Litmus perl module automates the process of generating a properly formatted xml result file and sending to the server. It provides an object-oriented interface to allow

The following example (perl) shows how to send your results to the server using the module:

use Test::Litmus;
  
  $t = Test::Litmus->new(-machinename => 'mycomputer',
  						 -username => 'user', 
  						 -password => 'pass',
  			# optional # -server => 'http://litmus.mozilla.org/process_test.cgi', 
  			# optional # -action => 'submit');
  			
  $t->sysconfig(-product => 'Firefox',
  				-platform => 'Windows', 
  				-opsys => 'Windows XP', 
  				-branch => 'Trunk', 
  				-buildid => '2006061314',
  				-locale => 'en-US');
  
  my $result = Test::Litmus::Result->new(
  			-testid => 27,
  			-resultstatus => 'pass', # valid results are 'pass'
  									 # or 'fail'
  			-exitstatus => 0,
  			-duration => 666,
  			-timestamp => 20051111150944,
  			-comment => 'optional comment here', # optional
  			-bugnumber => 300010, 				 # optional
  			-log => [Test::Litmus::Log->new(	 # optional
  						-type => 'STDOUT',
  						-data => 'foobar'),
  					 Test::Litmus::Log->new(
  					 	-type => 'Extensions Installed',
  					 	-data => 'log information here')]
  			);
  $t->addResult($result);
  # $t->addResult($someOtherResult);
  # etc...
  
  # add log information that should be linked with 
  # all results (i.e. env variables, config info)
  $t->addLog(Test::Litmus::Log->new(
  								-type => 'STDOUT',
  								-data => 'log data')); 
  
  my $res = $t->submit();
  
  # $res is 0 for non-fatal errors (some results were submitted), and 
  # undef for fatal errors (no results were submitted successfully)
  
  if ($t->errstr()) { die $t->errstr() }

Or, if you have already generated the xml result data manually, you may use the following to submit it to the server.

# $results contains the proper xml result data to send
use LWP::UserAgent;
my $ua = new LWP::UserAgent;
my $req = HTTP::Request->new(POST => 'http://litmus.mozilla.org/process_test.cgi');
$req->content_type('application/x-www-form-urlencoded');
$req->content('data='.$results);
my $res = $ua->request($req);

if (!$res->is_success) {
    # an error occurred in the submission 
}

my $content = $res->content;

if ($content =~ /^Fatal error/) {
    # a fatal error occurred
} elsif ($content =~ /^Error processing result for test (\d+)/) {
    # result $1 contained an error
}

Response

In the interest of keeping both the client and the server simple, the responses from the server are in plain text. After processing the results, the server will return (following the text/plain content type):

• The string 'ok' if all results were added to the database successfully.

• An error string beginning with the words 'Fatal error' if the result data cannot not be parsed or lacks required information.

• An error string beginning with the words 'Error processing result for test n' where n is the id of the test that resulted in the error. This line will be repeated for each result that contained an error.