14
edits
Compatibility/Meetings/Sync w Honza: Difference between revisions
30 May meeting minutes
(16 may meeting minutes) |
(30 May meeting minutes) |
||
| Line 1: | Line 1: | ||
== May 30 2023 == | |||
=== Top 100 testing (SV) === | |||
Note: Account login should be performed into TestRail before accessing the link | |||
We have started testing for top 100 sites using Test Rail. Test Run and Test Suites are available here: https://testrail.stage.mozaws.net/index.php?/projects/overview/72 | |||
Currently we are running Batch 3 after talking with the Desktop Team to avoid overlapping, which is due by the end of May. Since we have started later, Batch 3 will not be completed. | |||
Honza: So there are 3 batches? | |||
Raul: Yes, to avoid overlapping with QA Desktop. | |||
Honza: So 1 batch per month? what about the OS tested? | |||
Raul: Yes, 1 batch per month and we are testing on Windows 10 and Android but if we have more time we will test on Mac Os as well. | |||
Honza: What is the time gap between staring a new batch? | |||
Paul: So 1 batch per month but the gap between us and the QA Desktop is 2 months. | |||
Paul: Should we log the issues in Bugzilla or GitHub? | |||
Honza: We want to move everything in Bugzilla, so probably Bugzilla. | |||
Honza: After all this testing is done, where will the reports go? | |||
Paul: The webcompat bugs found by the Desktop QA team would end up on bugzilla. | |||
Honza: I see, normally we are working on a Knowledge base so any reports from webcompat would end up on bugzilla database in the future anyway. | |||
Raul: I have a question. We've already found 2 bugs based on our testing, submitted via webcompat.com reporter. Should we move them to Bugzilla from now on, under the Webcompatibility Product? | |||
Honza: So we have the Knowledge base Component, where we gather known compatibility issues [https://bugzilla.mozilla.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&product=Web%20Compatibility&query_format=advanced&list_id=16574680&component=Knowledge%20Base list], to help us understand issues better. This will be used to gather data for Telemetry and for the future release of our new reporter in Firefox. For now, if you find any issues when doing the OKR - Exploratory Testing for Top 100 sites, you can submit the issues using Bugzilla under the Desktop or mobile Component, as we will keep webcompat.com for now just for users. | |||
Honza: We should have all the issues logged into Bugzilla, as we are actively moving issues from webcompat to Bugzilla. | |||
Paul: Do we need to help with QA on the new simplified report flow? | |||
Honza: Yes, that's a good idea. We will keep you in the loop regarding the new Firefox Reporter, regarding updates and future tasks. | |||
Paul: Will this increase our workload, when the reporter will launched be in the release version? | |||
Honza: If that is the case, we will make a plan. I will share some documents with you to get a better idea of the new reporter. | |||
[https://docs.google.com/document/d/1uahBW-lW7XI87ZK6zaIWlkOciLcQdHhoPPRdpRXywyk/edit document_1] | |||
[https://docs.google.com/document/d/1T33NFqFr9a6n-P22BgpM_VXZWMTYRnYVtqLnTLH6dY4/edit#heading=h.6jdf2pjyoqjp document_2] | |||
Honza: The reports received now are from developers and advanced users, as most of the reports are received from the Nightly version, so they are pretty biased. We know that user reports are the best source of data. | |||
Paul: The key in that is that with so many users, we have also so many different configurations and set-ups. | |||
Honza: Only 2-3% finalize the webcompat reporting process due to the number of steps required to submit an issue. We will simplify the process for them, but it will be harder for us to replicate the issue. | |||
Calin: We could also add checkboxes to see if the users has add-ons/prefs active. | |||
Honza: Exactly. It will not be just URL description, but more, such as OS, graphic cards, etc, context required for enviroment set-up. Something very easy for users to check, to provide any usefull data. If will make it more complex, people will not do it. | |||
Raul: This might also be helpful for the other teams, like Add-ons team. | |||
Honza: Exactly. The Perfromnace team might be interested, the ETP team as well, based on what we can see in the data. If we are able to identify a signal in the reports, we can highlight it to the relevant team. | |||
Honza: Getting the reporting tool in the release version will take time, maybe months. We are thinking of introducing the simplified version in webcompat.com, like making the report imediatly, instead of going to the 7 steps process. it will go straight to data base. | |||
Paul: Is help needed from the QA to test the changes? | |||
Honza: That would be great. | |||
Paul: Will start to simplify the process now, and push the changes? | |||
Honza: We will keep the actual reporter, and we will be introducing new stuff to it, giving the reporter 2 option, the simplified tool or the old-complex reporter. The simplifications will be happening just on webcompat.com | |||
Paul: And the button will be added in the browser? | |||
Honza: Correct. | |||
Raul: Will all the versions benefit from the new reporter? | |||
Honza: Yes. Android is still under discussion. | |||
Raul: We can also test the changes that will be applied in webcompat.com | |||
Honza: Yes. Ksenia in actively working on this. | |||
Paul: Will the 2 variants, complex and simplified, still exist together? | |||
Honza: That will depend on the outcome. | |||
Paul: We should at least feed the users the simplified reporting first, and then feed them the complex one if they want to add more data. | |||
Honza: Agree. | |||
Honza: At somepoint, the reporting tool (new) will be the final path. Not sure what will happen to webcompat.com | |||
Honza: The biggest challenge will be to adapt and interpret the data. | |||
Paul: Will we have a bot like on GitHub that will cut down the noise from the new flux of reports? | |||
Honza: We are attempting to use the principle when we are using bots in Github. A way to identify top x pages, based on if the page is standard or if it is not supported by Firefox, to filter the noise, like we did similar for GitHub. | |||
=== Increase in issue engagement and number of issues (SV) === | |||
There has been a noticeable increase in the number of issues received this week. Most likely, as seen around the web, dark reader is causing issues with Firefox. | |||
Also, some users have started to engage with our reports, and sometimes to interfere with our diagnosis process. | |||
E.g: | |||
https://github.com/webcompat/web-bugs/issues/122797 | |||
https://github.com/webcompat/web-bugs/issues/122772 | |||
Honza: What are they commenting about, are they providing usefull info? | |||
Raul: Some yes but most of them just clone our reponses. | |||
Honza: The team will take care of the ones that irelevant. You might want to communicate with the ones providing usefull information. | |||
Paul: So we already received bugs regarding dark reader? Because I observed that it started to mess many websites (I'm using it by the way). | |||
Raul: Calin observed that some issues are fixed if Dark Reader is disabled. | |||
== May 16 2023 == | == May 16 2023 == | ||
| Line 88: | Line 210: | ||
Raul: Will have it ready by then. | Raul: Will have it ready by then. | ||
=== Updates- Top 100 (Honza) === | === Updates - Top 100 (Honza) === | ||
Paul: We are still waiting for the feedback from the team, regarding what would be important in the webcompat area, what we should focus on. Maybe be can use the same list as the desktop team or do another round of checks to see if the list entries have changed. We will take a look to see what the list looks now vs how it should look with the current top sites. | Paul: We are still waiting for the feedback from the team, regarding what would be important in the webcompat area, what we should focus on. Maybe be can use the same list as the desktop team or do another round of checks to see if the list entries have changed. We will take a look to see what the list looks now vs how it should look with the current top sites. | ||