AUS:Tests: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Questions =
[[AUS|« back to AUS main page]]
* Do we start over?
* What can we move over from the bash tests?
* Do we just keep things in bash and wrap the new stuff?
* Where should we plug in the existing updates.cfg?
* How would we piggyback the actual update application onto a new set of .py tests?  Doable?
 
= TODO =
* Decide how we will integrate the existing release tests with pyFit-based tests
* Start setting up very simple tests and utilize pyFit to store our 'expected results'
** We would have to generate some mock build data, and use an aus2 test instance to help generate some of the 'expected results'
** Start building off of this to eventually cover all of our possible combinations of inputs/outputs
* Develop a script to finally retrieve AUS XML data based on passed inputs/fixtures then compare to expected results
* Think about ways to display results or provide reporting capabilities down the road?  Logging?
 
= AUS Flow =
To learn a little about what logic determines what XML is displayed, see the [http://people.mozilla.org/~morgamic/aus-partner.jpg AUS flow chart].
 
= Test Assumptions =
Sets of rules we can use to make our tests easier:
* For a given product, platform and locale, the output for all possible combinations having buildID more than one-off the latest will be identical
* One-off buildID offerings will have an added partial update that points to the latest build.


= Existing Tools =
= Existing Tools =
These need to be consolidated into a more complete single testing suite:
These need to be consolidated into a more complete single testing suite:
* [http://khan-vm.mozilla.org:8080/VerifyAus?test AUS acceptance tests]
* [http://users.ox.ac.uk/~clar0239/upwatch/staging/ cf's output comparison]
* [http://users.ox.ac.uk/~clar0239/upwatch/staging/ cf's output comparison]
** Good visualization idea, good coverage, but only for nightlies
** Good visualization idea, good coverage, but only for nightlies
Line 31: Line 11:
** Has better coverage per-test -- downloads .mar and compares hashes.  Notifies if there is a 404 or hash mismatch.
** Has better coverage per-test -- downloads .mar and compares hashes.  Notifies if there is a 404 or hash mismatch.
** l10n tests
** l10n tests
= Inputs =
* uri version
* product
* version
* build
* platform
* locale
* channel
= Outputs =
* See [[Software_Update:updates.xml_Format|Software Update XML format definition]]
Example:
<?xml version="1.0"?>
<updates>
    <update type="minor" version="1.5.0.7" extensionVersion="1.5.0.7" buildID="2006083105" >
        <patch type="complete" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-08-31-05-mozilla1.8.0/firefox-1.5.0.7.en-US.mac.complete.mar" hashFunction="md5" hashValue="90f58b8836f288d42e2c14ea5ba3c71e" size="15408802"/>
        <patch type="partial" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-08-30-05-mozilla1.8.0/firefox-1.5.0.7.en-US.mac.partial.2006083005-2006083105.mar" hashFunction="MD5" hashValue="712b418e5c300be941515d5f75eb167f" size="6859"/>
    </update>
</updates>
= AUS Flow =
To learn a little about what logic determines what XML is displayed, see the [http://people.mozilla.org/~morgamic/aus-partner.jpg AUS flow chart].
= Test Assumptions =
Sets of rules we can use to make our tests easier:
* For a given product, platform and locale, the output for all possible combinations having buildID more than one-off the latest will be identical
* One-off buildID offerings will have an added partial update that points to the latest build.

Latest revision as of 07:44, 24 January 2007

« back to AUS main page

Existing Tools

These need to be consolidated into a more complete single testing suite:

Inputs

  • uri version
  • product
  • version
  • build
  • platform
  • locale
  • channel

Outputs

Example: 

<?xml version="1.0"?>
<updates>
    <update type="minor" version="1.5.0.7" extensionVersion="1.5.0.7" buildID="2006083105" >
        <patch type="complete" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-08-31-05-mozilla1.8.0/firefox-1.5.0.7.en-US.mac.complete.mar" hashFunction="md5" hashValue="90f58b8836f288d42e2c14ea5ba3c71e" size="15408802"/>
        <patch type="partial" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-08-30-05-mozilla1.8.0/firefox-1.5.0.7.en-US.mac.partial.2006083005-2006083105.mar" hashFunction="MD5" hashValue="712b418e5c300be941515d5f75eb167f" size="6859"/>
    </update>
</updates>

AUS Flow

To learn a little about what logic determines what XML is displayed, see the AUS flow chart.

Test Assumptions

Sets of rules we can use to make our tests easier:

  • For a given product, platform and locale, the output for all possible combinations having buildID more than one-off the latest will be identical
  • One-off buildID offerings will have an added partial update that points to the latest build.
Retrieved from "https://wiki.mozilla.org/index.php?title=AUS:Tests&oldid=47350"