Compatibility/Mobile/2015-04-07

From MozillaWiki
Jump to: navigation, search

Previous 2015-03-31

Minutes

Automated Regression Testing (mike/hallvord?)

Is it possible to collaborate on a tool between Web Compat and Rel Man for automated regression testing? What are the requirements that each of us have? Would this be better discussed in person in Whistler?

  • miketaylr: we invited lawrence to join us. Web Compatibility and Release Management are both potentially interested in tracking a set of sites for regression. For WebCompat day to day, For Release Management maybe more specific releases. What would the requirements from RM.
  • lawrence: Are we breaking any Web content? Have we introduced new regressions, we broke them or the site broke us. Could be changed in layout, security, graphics, etc. And then gather the information on who we should contact to fix it. Desktop and Mobile (Android). If we break something, the second question is then how bad do we break the Web. 1st one is Can I release 2. Should we fix it? 3. How do I convince them to fix it?
  • miketaylr: One of our questions, which sites do we track? We need a system with a flexibility where we can grow the system.
  • hallvord: We have two sorts of problems: Things we know which are broken. Things more explorative where we want to discover if it breaks. We need some kind of dictionary for equivalence of error messages across browsers. CSS Comparisons. Error tracking. HTTP redirects. These tests need to run in two engines simultaneously to avoid disparities. Change of content (widgets, ads), make it harder.
  • miketaylr: One recent example is the ClojureScript bug, it would be good to know in advance how much we will break Web sites.
  • hallvors: As a requirement we need to have something where we can modify the environment before testing.
  • miketaylr: There's a potential for overlap. And it looks also like a huge undertaking.
  • hallvors: Using marionette or slimerJS is not that complicated, but running two engines simultaneously and they have their own differences. PhantomJS/SlimerJS is a good start.
  • miketaylr: 2 engines interesting for RM.
  • lawrence: as a second priority, it is interesting. If we do a change, that is already broken in Chrome, it's kind of easier for not being so worried about breaking things.
  • miketaylr: It would be ideal to build something small, that is modular so we could easily add to it.
  • karl: There are parts of the infrastructure that are very similar. The things that are different, for Release Management it might be a different version of Firefox, for Web Compatibility it might also be a different Engine. If we are able to design a system in a way to isolate the task we should be able to create a tool that is useful for everyone.
  • hallvors: The tool that could help us to isolate which gecko has been creating an issue.
  • lawrence: We do not have the time to go to each site.
  • hallvors: it would be good to be able to store the content of the page, but it starts to create a lot of storage. Like the tool that existed at Opera.
  • miketaylr: As a tool, it was becoming unintelligible. Maybe we should write down our requirements. We also have compatipede 1. There is a version 2 which is not opensource, but too uncertain for now. I think we should maybe take a break to think about this and regroup later.
  • karl: I think Telemetry has a sort of connection with all of this, in terms of discovering sites that will have issues with certain things.
  • hallvord: Do you get the URL?
  • miketaylr: you don't but there's progress for doing that.
  • lawrence: There are two versions of data collections. There's a differenciation because of regulations in each countries, and they don't collect the exact same thing and the same platforms.
  • miketaylr: there is a way. It's possible.
  • lawrence: it's tricky. Tracking the features instead of the sites, we will know how many % will be impacted by a change.
  • hallvord: The requirements for RM and WC are pretty much the same.
  • lawrence: I would like us to try to pick up one thing and to grow it.
  • karl: There's probably already pieces of the puzzle that exist in Mozilla, in terms of infrastructure.
  • miketaylr: We should meet in Whistler about this.
  • hallvord: it would be nice to have a decision before Whistler.
  • lawrence: Maybe Telemetry could be a way to start it, and if we need URLs in reports, we should push for it.
  • hallvord: The probe is useful and easy. but an example like the ClojureScript is a really small number of Web sites. I looked at the list of sites that were declared.
  • karl: We have a list of candidate probes. URL: <WIKI-PROBE>
  • miketaylr: ACTION ITEMS?
  • karl: I like the idea of lawrence, to stay simple. We need a prototype of a small actionable thing, that we can come to Whistler to show a working tool. We have a big shopping list of things we would like to do, but here is working.
  • hallvord: I think I'm a bit past that stage.
  • miketaylr: You should explain and advertise the tools already done.
  • lawrence: Data useful for other teams could be a first step. The earlier we find the bug the cheaper it will be to fix the things. Commit time would be good.
  • ACTION: hallvord to extract Web Compatibilty data through a dashboard which could be useful for other teams (possibly via github.com/mozilla/corsica - office screens)
  • ACTION: hallvord to explain the list of tools for Web Compatibility which are doing survey and data collection.

Agenda Web Compat in Whistler 2015 (karl)

The All hands work week meeting in Whistler Canada is very close we need to prepare an agenda so we are maximizing the time there.

  • See: https://wiki.mozilla.org/Compatibility/Mobile/2015-06-work-week
  • karl: Please look carefully at the agenda. The richness of the discussions are partly dependent on the topic bank and help us shape the meetings we have with other parties. The earlier the better. ACTION ITEM: add topics!!1111!1
  • lawrence: It helped my team to book meetings well in advance.

Heads Up

Broken Voices of the Web


Web Compatibility Progress

FIXED (no DUPLICATE)

NEW

Retrieved from "https://wiki.mozilla.org/index.php?title=Compatibility/Mobile/2015-04-07&oldid=1066742"