TestEngineering/Web/Automation/Baseline: Difference between revisions

Latest revision as of 08:12, 19 January 2018

This page is being used to indicate a baseline for our automation and how our projects stack up.

Current

Project	README	Dev Workflow	Python 3	pyup.io	ActiveData	Treeherder	flake8-isort
Bouncer	complete	not started	not started	complete	complete	complete	complete
FoxPuppet	complete	not applicable	complete	complete	not started	not started	complete
FxAPOM	complete	not applicable	complete	complete	complete	complete	complete
Bedrock	not applicable	complete	not started	not started	not started	not started	not applicable
Mozillians	complete	not started	not started	complete	complete	complete	complete
PyPOM	complete	not applicable	complete	complete	not applicable	not started	complete
Snippets	complete	complete	not started	complete	complete	complete	complete
Socorro	complete	complete	not started	not started	complete	complete	complete
Stub Attribution	complete	not started	not started	complete	complete	complete	complete
Treeherder	complete	in progress	not started	complete	complete	complete	in progress

Completed

Enable requires.io
Enable GitHub trigger for Jenkins
Switch from unittest-zero to native asserts
Migrate from pytest-mozwebqa to pytest-selenium
Migrate to Jenkins pipeline
Migrate to Tox
Use email-ext for pipeline notifications
Remove implicit waits
Switch to declarative pipelines
Migrate to PyPOM

@@ Line 1: / Line 1: @@
 This page is being used to indicate a baseline for our automation and how our projects stack up.
+== Current ==
 {| class="wikitable sortable"
 |-
-! Project !! pytest-selenium !! native asserts || no implicit waits || README || requires.io || GitHub trigger
+! Project !! README || Dev Workflow || Python 3 || pyup.io || ActiveData || Treeherder || flake8-isort
 |-
-| [https://github.com/mozilla/Addon-Tests/ AMO]
+| [https://github.com/mozilla-services/go-bouncer Bouncer]
+| style="background:lightgreen;text-align:center;" | complete
+| style="background:pink;text-align:center;" | not started
 | style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 |-
-| [https://github.com/mozilla/affiliates-tests/ Affiliates]
+| [https://github.com/mozilla/FoxPuppet/ FoxPuppet]
+| style="background:lightgreen;text-align:center;" | complete
+| style="background:lightgray;text-align:center;" | not applicable
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
+| style="background:pink;text-align:center;" | not started
 | style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
+|-
+| [https://github.com/mozilla/fxapom/ FxAPOM]
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgreen;text-align:center;" | complete
+| style="background:lightgray;text-align:center;" | not applicable
-|-
-| [https://github.com/mozilla/bidpom BIDPOM]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 |-
-| [https://github.com/mozilla/bouncer-tests/ Bouncer]
+| [https://github.com/mozilla/bedrock/ Bedrock]
 | style="background:lightgray;text-align:center;" | not applicable
 | style="background:lightgreen;text-align:center;" | complete
+| style="background:pink;text-align:center;" | not started
+| style="background:pink;text-align:center;" | not started
+| style="background:pink;text-align:center;" | not started
+| style="background:pink;text-align:center;" | not started
 | style="background:lightgray;text-align:center;" | not applicable
-| style="background:pink;text-align:center;" | not started
-| style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgreen;text-align:center;" | complete
 |-
-| [https://github.com/mozilla/fxapom/ FxAPOM]
+| [https://github.com/mozilla/mozillians-tests Mozillians]
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:pink;text-align:center;" | not started
 | style="background:pink;text-align:center;" | not started
+| style="background:lightgreen;text-align:center;" | complete
+| style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 |-
-| [https://github.com/mozilla/hello-tests/ Hello]
+| [https://github.com/mozilla/PyPOM/ PyPOM]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/hello-tests/pull/16 in review]
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/hello-tests/pull/16 in review]
+| style="background:lightgray;text-align:center;" | not applicable
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/hello-tests/pull/15 in review]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
-|-
+| style="background:lightgray;text-align:center;" | not applicable
-| [https://github.com/mozilla/marketplace-tests Marketplace]
-| style="background:pink;text-align:center;" | not started
-| style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 | style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 |-
-| [https://github.com/mozilla/mcom-tests/ Mozilla.org]
+| [https://github.com/mozmeao/snippets-service/tree/master/tests Snippets]
-| style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
-|-
-| [https://github.com/mozilla/bedrock/ Bedrock]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgray;text-align:center;" | not applicable
-| style="background:lightgray;text-align:center;" | not applicable
-| style="background:lightgray;text-align:center;" | not applicable
 |-
-| [https://github.com/mozilla/mozillians-tests Mozillians]
+| [https://github.com/mozilla-services/socorro/tree/master/e2e-tests Socorro]
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgreen;text-align:center;" | complete
-|-
-| [https://github.com/mozilla/moztrap-tests MozTrap]
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
 | style="background:pink;text-align:center;" | not started
-| style="background:lightgreen;text-align:center;" | complete
 | style="background:pink;text-align:center;" | not started
-|-
-| [https://github.com/mozilla/mozwebqa-examples Web QA Examples]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/mozwebqa-examples/pull/4 in review]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgray;text-align:center;" | not applicable
 |-
-| [https://github.com/mozilla/oneanddone-tests One and Done]
+| [https://github.com/mozilla/stubattribution-tests Stub Attribution]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/oneanddone-tests/pull/44 in review]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:pink;text-align:center;" | not started
@@ Line 108: / Line 85: @@
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
-|-
-| [https://github.com/mozilla/snippets-tests Snippets]
-| style="background:lightgray;text-align:center;" | not applicable
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:lightgray;text-align:center;" | not applicable
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
 |-
-| [https://github.com/mozilla/Socorro-Tests/ Socorro]
+| [https://github.com/mozilla/treeherder/tree/master/tests/jenkins Treeherder]
-| style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
+| style="background:lightyellow;text-align:center;" | in progress
 | style="background:pink;text-align:center;" | not started
 | style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
-|-
-| [https://github.com/mozilla/sumo-tests Support]
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
-| style="background:pink;text-align:center;" | not started
-| style="background:lightgreen;text-align:center;" | complete
-| style="background:pink;text-align:center;" | not started
-|-
-| [https://github.com/mozilla/treeherder-tests Treeherder]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/treeherder-tests/pull/26 in review]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/treeherder-tests/pull/24 in review]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/treeherder-tests/pull/26 in review]
-| style="background:lightyellow;text-align:center;" | [https://github.com/mozilla/treeherder-tests/pull/22 in review]
 | style="background:lightgreen;text-align:center;" | complete
 | style="background:lightgreen;text-align:center;" | complete
+| style="background:lightyellow;text-align:center;" | in progress
 |}
+== Completed ==
+* Enable [https://requires.io/ requires.io]
+* Enable GitHub trigger for Jenkins
+* Switch from [https://github.com/AutomatedTester/unittest-zero unittest-zero] to [http://pytest.org/latest/assert.html native asserts]
+* Migrate from [https://pypi.python.org/pypi/pytest-mozwebqa pytest-mozwebqa] to [http://pytest-selenium.readthedocs.org/en/latest/ pytest-selenium]
+* Migrate to Jenkins pipeline
+* Migrate to Tox
+* Use [https://plugins.jenkins.io/email-ext email-ext] for pipeline notifications
+* Remove implicit waits
+* Switch to declarative pipelines
+* Migrate to [https://github.com/mozilla/PyPOM PyPOM]