Automationeers Assemble

From MozillaWiki
Jump to: navigation, search

Automationeers Assemble is a regular gathering featuring a guest and topic relevant to automation engineering.

How to attend

Many gatherings are live streamed on YouTube

You can also join our Zoom room: 487-987-164 or dial-in using your local number

Invitations are sent to the automationeers-assemble Google Group.

Keeping in touch

Upcoming events

Cross Device Automation

πŸ“… TBA


Past events

Device Lab Management

πŸ“… May 14th, 2020 (18:00 UTC)

πŸŽ₯ watch the video

Sakari joined us from BitBar to share the challenges of maintaining a comprehensive mobile device lab. He covered a number of topics such as the space requirements and racking, network connectivity, device batteries and charging, monitoring device health, and remote access.


πŸ“… April 16th, 2020 (18:00 UTC)

πŸŽ₯ watch the video

Patrick provided a walkthrough of WebPageTest.

Your Tests Lack Vision

πŸ“… March 18th, 2020 (18:00 UTC)

πŸŽ₯ watch the video

Automation has come a long way in assisting with regression testing efforts. Teams worldwide are successfully running hundreds of functional regression tests at every check-in. While this provides a great source of confidence, critical regression bugs are still missed using this approach. That’s because these tests can only assert on what their human programmer asks them to. Additional errors with functionality, UX, and usability often go uncaught using today’s most common test automation techniques.

For this reason, the top companies in all sectors of the industry are turning to visual validation. Visual validation is a relatively new concept that can be used to enhance existing automated tests and provide an easy way to perform those difficult checks for things like UX, localization, usability, responsive design, and cross-device testing.

In this talk, you’ll learn how visual validation works, see a live integration into an existing test code base, and discuss the pros and cons of using various visual validation techniques.

Wikipedia Quality of Experience

Gilles Duboc

πŸ“… September 12th, 2019 (15:00 UTC)

πŸŽ₯ watch the video

Gilles joined us from the Wikimedia Foundation to talk about his involvement in a large-scale study of Wikipedia users' quality of experience.


Kyle Lahnakoski

πŸ“… August 29th, 2019 (19:00 UTC)

πŸŽ₯ watch the video

The ActiveData service, is a public-facing centralized query service for Mozilla data that is generally gathered from Taskcluster. It includes various data points on things like code coverage, performance metrics, test failure classifications, and the list goes on. Although multiple implementation possibilities exist for a centralized database, they each have their pros & cons, and ActiveData takes advantage of these. In this talk, the components that make up the ActiveData service, why they were chosen, and how they work together will be discussed. The data format and the ActiveData query language will then be presented, as well as some examples of what ActiveData is capable of. Finally, some points on the future directions of this service will be highlighted.

Interoperability Testing for Firefox

πŸ“… July 18th, 2019 (19:00 UTC)

πŸŽ₯ watch the video

Kimblerly Sereduck joined us to share details on an infrastructure she's developing to automate the detection of crashes in Firefox caused by third-party software, such as antivirus programs.

Optimizing with Webhint

πŸ“… May 9th, 2019 (20:00 UTC)

πŸŽ₯ watch the video

Brendyn Alexander and John Jansen join us from Microsoft to provide a brief history and overview of webhint focusing on how it can help web developers at various points in the develop-debug-deliver pipeline optimize the sites and apps they build.

Performance Testing in Chrome

πŸ“… April 25th, 2019 (19:00 UTC)

πŸŽ₯ watch the video

Annie Sullivan joins us to provide an overview of Chrome's performance testing framework and tooling.

Customizing your pytest test suite

πŸ“… March 12th, 2019 (15:00 UTC)

πŸŽ₯ watch the video

πŸ“ guest resources

Raphael provides an introduction into automated testing in Python with pytest, a popular testing framework developed and maintained by a thriving community of volunteers. After looking into how to develop automated tests with pytest, he dives into writing plugins for pytest and explore ideas for potential test suite customizations, that help you and your team manage a growing test suite.

Task Configuration at Scale

πŸ“… February 14th, 2019 (20:00 UTC)

πŸŽ₯ watch the video

πŸ“ guest resources

Learn about the taskgraph module and how Mozilla tries to keep task configuration flexible, maintainable and easy to use at a large scale.

Accessibility Testing with Bubo Eye

πŸ“… January 17th, 2019 (20:00 UTC)

πŸŽ₯ watch the video

Bubo Eye is a WIP Firefox extension that will improve and automate image content accessibility for screen reader users. Screen readers can access the brief blurb that describes an image with the use of the alt attribute. Bubo Eye will check for text in memes, diagrams, comics, documents, scan it and make that content available to the screen reader user.

Adventures in Android Device Testing

πŸ“… November 14th, 2018 (16:00 UTC)

πŸŽ₯ watch the video

Chris Hartjes shares his experiences in evaluating and prototyping testing designed to replace a manual testing process for verifying features of Mozilla's push notification service.

Performance at Wikimedia

πŸ“… October 25th, 2018 (20:00 UTC)

πŸŽ₯ watch the video

πŸ“ guest resources

Peter is the author of the web performance tool He’s a huge fan of open source, and works on the performance team at the Wikimedia Foundation. He’ll be sharing with us the work they’re doing, what they aim for, how they improved finding regressions, and where they want to be in the next year. Peter is also looking forward to answering your questions.


Steven Englehardt

πŸ“… September 13th, 2018 (15:00 UTC)

πŸŽ₯ watch the video

OpenWPM is a web privacy measurement framework which makes it easy to collect data for privacy studies on a scale of thousands to millions of websites. OpenWPM is built on top of Firefox, with automation provided by Selenium. It includes deep instrumentation hooks within a Firefox extension to record HTTP request and response data, Javascript calls, and browser storage access.

OpenWPM is currently a Princeton University project in the process of being migrated to Mozilla. Source and docs are available here.


Rob Wood

πŸ“… September 13th, 2018 (15:00 UTC)

πŸŽ₯ watch the video


Raptor is a new Firefox performance test framework currently under development. It will be used for the next generation of desktop page-load and benchmark tests, and is based on a browser web extension which is cross-browser compatible. Raptor is currently running on mozilla-central as tier 3 on both Firefox and Google Chrome.


Matt Wobensmith

πŸ“… July 19th, 2018 (18:00 UTC)

πŸ“ guest resources

Iris is a new tool that solves some traditional problems in testing UI-driven applications. It uses on-screen bitmap pattern matching, combined with mouse/keyboard input, to create complex interactions normally performed by manual testers. This, along with a simplified test writing model, aims to rapidly automate potentially thousands of test cases that might otherwise not be automatable.

The project is currently in a pre-beta state, but has over 50 test cases. A projected launch of a 1.0 project is slated before the end of the year. The repo and docs are currently under development, but will be contributor-friendly by launch time.