EngineeringProductivity/Projects/Everything

Project Table

We have many projects in different states. This table below should highlight what is being done and upcoming work in the future. If you want to get involved, look for a higher [Friendly] score!

Priority is an arbitrary ranking based on how important it is for us to complete a milestone or the project. A Priority 1 is something that others are depending on or something we have promised to get done in the short term. A priority 2 is something we probably will spend some time on and be happy to help with. A priority 3 is something we want to do and will probably tackle it in the near future!

Project Name	Link to Details/Docs	Contact Info	Friendly (1-10)	Priority (1-3)
Alert Manager	[AlertManager Wiki]	dminor,jmaher	6	2
Autoland	[Autoland Wiki]	dminor	5	1
Autophone	[AutoPhone Wiki]	bc	2	2
Bugzilla	[Bugzilla ReadTheDocs]	dkl,dylan,glob,mcote	4	2
BugzFeed	[BMO Change Notification]	mcote	4	2
Bugzilla/ESCluster	[BMO Elastic Search]	ekyle	3	3
Charts	[Github for Charts]	ekyle	1	3
Datazilla	[Datazilla ReadTheDocs]	camd,mdoglio,wlach	4	3
DevTools Harness	[Github Repo]	ted	2	1
dzAlerts	[Alerts Wiki]	ekyle	4	2
Eideticker	[Eideticker Wiki]	davehunt,wlach	3	3
m21s	[m21s Wiki]	ahal,chmanchester,whimboo	5	1
Marionette	[Marionette Wiki]	ato,AutomatedTester,jgriffin	7	1
Mozharness	[Mozharness Wiki]	armenzg	4	2
MozReview	[MozReview Wiki]	mcote	4	1
OrangeFactor	[OrangeFactor Wiki]	jgriffin,mcote	3	3
Ouija	[Ouija Wiki]	dminor,jmaher	4	3
Pulse	[Pulse Wiki]	mcote	4	3
Structured Logging	[Structured Logging ReadTheDocs]	ahal,chmanchester,jgraham	1	3
Talos	[Talos Wiki]	jmaher	5	3
TBPL	[TBPL Wiki]	edmorley	4	2
Test Informant	[Test Informant Wiki]	ahal	3	2
Treeherder	[Treeherder Wiki]	camd,edmorley,mdoglio	7	1
Web-Platform-Tests	[Test the Web Forward] [WPT ReadTheDocs]	jgraham	2	3

Motivation

This page is to help diagram all the various things the A*Team has built and continue to support.

The plan is to draw high level components/systems, and links between them, so it is easier to see how all the parts work together. Hopefully this diagram will be clickable so the reader can get more detail on each of the components.

Something like what RelEng has, but without all the internals.

History (blah blah blah..)

My name is Kyle Lahnakoski, and I joined the A*Team in April 2013.

When I started, the plan was to improve the alerting mechanism in Datazilla. Jmaher and jeads were kind enough to give me an overview of the various systems surrounding Datazilla. I drew my notes and diagrams on paper, and for months I carried my "secret decoder" so I could talk reasonably intelligently about the various A*Team systems and how they connected.

It is late June 2014 now, and my notes are long lost. I recently had a request from another team's member (:avih) about how dzAlerts fits into the greater ecosystem. In a recent meeting jgriffin mentioned it would be nice to have a summary of all the great things the A*Team works on and supports.

I would like to draw another not-so-secret decoder, but this time with everything!

Please help by adding the A*Team systems to The List below

Example

Use the EMPTY TEMPLATE to add more entries. Include ones you know about, even if you can not fill them.

EXAMPLE

Docs - some wiki, or read the docs to learn more
Code - link to code, if it makes sense
Contact - for ekyle to contact if he has questions
Inputs -
- automated resources
- services consumed
Outputs/Services
- automated services provided
- dashboards (and teams that consume them)

EMPTY TEMPLATE

Docs -
Code -
Contact -
Inputs
Outputs/Services

The List

AlertManager

Docs -https://github.com/jmaher/alert_manager/
Code - https://github.com/jmaher/alert_manager/
Contact' - jmaher@mozilla.com
Inputs
- mozilla.dev.tree-management alerts from graph server
Outputs/Services
- webUI for organizing and taking action on these alerts.

Autoland

Docs - https://wiki.mozilla.org/Auto-tools/Projects/Autoland
Code - https://github.com/dminor/autoland
Contact - [:dminor]
Inputs
- try jobs via pulse
- job status via relengapi
Outputs
- autoland comments posted to bugzilla
- patches landed via releng's 'hg transplant'

Autophone

Docs - https://wiki.mozilla.org/Auto-tools/Projects/AutoPhone
Code - https://github.com/mozilla/autophone/
Contact -
Inputs
Outputs/Services
- http://phonedash.mozilla.org/ (used by who?)

Bugzilla

Docs - https://bugzilla.readthedocs.org
Code - https://git.mozilla.org/?p=webtools/bmo/bugzilla.git
Contact - Mark Cote, Byron Jones, David Lawrence, Dylan Hardison
Inputs
- Web UI
- WebServices API (REST, XMLRPC, JSONRPC)
Outputs/Services
- https://bugzilla.mozilla.org (bug reporters, developers, etc)
- WebServices API (For REST https://bugzilla.mozilla.org/rest)

BugzFeed

Docs - BMO/ChangeNotificationSystem
Code - https://github.com/mozilla/bugzfeed
Contact - mcote@mozilla.com
Inputs
- BMO via Pulse
Outputs/Services
- WebSocket servers:
  - bugzfeed.mozilla.org
  - bugzfeed-dev.allizom.org

Bugzilla/ES Cluster

Docs - https://wiki.mozilla.org/BMO/ElasticSearch
Code - https://github.com/klahnakoski/Bugzilla-ETL
Contact' - klahnakoski@mozilla.com
Inputs -
- Bugzilla (direct database access)
Outputs/Services
- https://esfrontline.bugzilla.mozilla.org:443/public_bugs/bug_version

Buildbot

Docs -
Code -
Contact -
Inputs
- hg.mozilla.org?
Outputs/Services
- builder slaves
- tester slaves
- https://wiki.mozilla.org/ReleaseEngineering/BuildAPI ??
- where do buildbot results go, a file? picked up by TBPL

C++ Code Coverage

Docs - https://bugzilla.mozilla.org/show_bug.cgi?id=890116
Contact - [:ahal]

charts.mozilla.org

Docs -
Code - https://github.com/mozilla/charts
Contact' - klahnakoski@mozilla.com
Inputs -
- BMO/ES Cluster (https://esfrontline.bugzilla.mozilla.org:443/public_bugs/bug_version)
Outputs/Services
- http://charts.mozilla.org (for management)

Datazilla

Docs - https://datazilla.readthedocs.org/en/latest/
Code - https://datazilla.readthedocs.org/en/latest/
Contact - See Treeherder
Inputs
- data pushed from Talos (see Graph Server)
Outputs/Services
- charts at https://datazilla.mozilla.org
- raw results at https://datazilla.mozilla.org/talos/refdata/objectstore/json_blob

DevTools Harness

Docs - Needs project page!
Contact - [:ted.mielczarek]

dzAlerts

Docs - https://wiki.mozilla.org/Auto-tools/Projects/Alerts
Code - https://github.com/klahnakoski/datazilla-alerts
Contact' - klahnakoski@mozilla.com
Inputs
- Datazilla web service (https://datazilla.mozilla.org/talos/refdata/objectstore/json_blob)
- Eideticket web service (http://eideticker.mozilla.org)
Outputs/Services
- fxos-perf-alerts@mozilla.org (b2g sheriffs?)

Eideticker

Docs - https://wiki.mozilla.org/Project_Eideticker#Documentation
Code - https://github.com/mozilla/eideticker
Contact - William Lachance, Dave Hunt
Inputs
- Nightly/inbound b2g and android builds.
Outputs/Services
- Eideticker dashboard (http://eideticker.mozilla.org)

m21s

Docs - https://wiki.mozilla.org/Auto-tools/Projects/m21s
Code - see project page
Contacts - [:chmanchester][:ahal][:whimboo]
Inputs
- Mozmill release tests
Outputs
- 'Firefox Greenlight Tests' in Marionette, reporting to Treeherder

Marionette

Docs - https://wiki.mozilla.org/Auto-tools/Projects/Marionette
Code -
Contact - [:AutomatedTester][:ato]
What - Mozilla's native WebDriver implementation

Mozharness

Docs - https://wiki.mozilla.org/Auto-tools/Projects/Mozharness
Code - https://hg.mozilla.org/build/mozharness/
Contact - [:armenzg]
Inputs
- command-line invocation via buildbot or locally
Outputs/Services
- build/test logs consumed by buildbot, Treeherder, TBPL

MozReview (Review Board)

Docs - Auto-tools/Projects/CodeReviewTool
Code - various pieces of https://hg.mozilla.org/hgcustom/version-control-tools; see docs for more.
Contact - mcote@mozilla.com
Inputs
- BMO
- hg.mozilla.org
Outputs/Services
- BMO

OrangeFactor

Docs - Auto-tools/Projects/OrangeFactor
Code - https://hg.mozilla.org/automation/orangefactor/
Contact - :jgriffin, :mcote
Inputs
- buildbot messages (using pulse)
- build logs
- comments from TBPL
- bugzilla bug data
Outputs/Services
- https://brasstacks.mozilla.com/orangefactor/
- email status [War on Orange]

Talos

Docs - https://wiki.mozilla.org/Buildbot/Talos
Code - https://hg.mozilla.org/build/talos
Contact - jmaher@mozilla.com
Inputs
- build for desktop or android
- commandline (done via mozharness now) to specify which test(s) to run
Outputs/Services
- log file with raw data
- uploaded summarization of data to graph server
- uploaded raw data to datazilla

Can be referred to as what is run by buildbot or by tbpl since this is the only performance test run on our per revision CI system. Other tools are run at a different cadence.

TBPL (for the sake of history)

Docs - Sheriffing/TBPL
Code - https://hg.mozilla.org/webtools/tbpl/
Contact - Sheriffing/TBPL#Developer_Contacts
Inputs
Outputs/Services

Test Informant

Docs - https://wiki.mozilla.org/Auto-tools/Projects/Test-Informant
Code - See project page
Contact - [:ahal]
Inputs
- build notifications via pulse
- test manifests in tests.zip
Outputs
- reports at e.g., http://brasstacks.mozilla.com/testreports/weekly

Treeherder

Docs - Auto-tools/Projects/Treeherder
Code - Auto-tools/Projects/Treeherder#Source_and_Docs
Contact - [:camd][:mdoglio][:edmorley]
Inputs
- Pushlog from hg.mozilla.org via json-pushes
- Build/tests from Buildbot via buildapi
- Direct pushes via ``treeherder-client``
- Build/tests from Taskcluster (under development)
Outputs/Services
- https://treeherder.mozilla.org/
- https://treeherder.mozilla.org/api/

web-platform-tests

Docs - https://wptrunner.readthedocs.org/en/latest/, Needs project page?
Code
Contact - [:jgraham]