QA/Fenix/TopSites

From MozillaWiki
< QA
Jump to: navigation, search

Firefox Preview Top Sites Test Plan

  • TestRail test plan: Testrail
  • Testrail Testrail - on 10/4 Nightly build with Xiaomi Mi 8 Lite (Android 9)
  • Test Plan TestPlan - on 01/20, and 01/21 Nightly builds, with Google Pixel (Android 10), and Sony Xperia Z2 (Android 6.0.1)

Mobile Testing

The Softvision Mobile QA team will be testing the top mobile sites on Firefox Preview, a list of prioritized locales and reporting bugs as they find them. We made a plan to approach this request:

    • QA will research a list with the most common sites from the US, Europe (UK, Germany, France), India, China, Hong Kong, Russia.
    • QA will write the tests after the research.
    • QA will classify the sites in some criteria: social media sites, e-commerce sites, news sites, media playback sites.
    • QA will check some sites daily, as much as time permits, until all sites are covered.
    • QA will do a regression check after each GV update on each website.
    • QA will check Alexa's 50 top sites monthly and see if there are rank modifications or new pages introduced (the tests will be updated).
    • QA will file bugs, then verify fixed bugs, when necessary.

Test Plan

In general, we want to test a site in the most useful ways it can be tested, beyond looking at the home page, i.e., play and pause video, create an account and log in, add and remove an item to/from a cart, open a menu and select items, upvote a post, etc. The plan is to start exploratory testing with the US locale and make notes of what is and isn't useful (or possible) to test. From that, we'll create a test plan of the remaining locales based on the types of sites and their related common interactions.

NSFW Content

Some sites tested may be adult or pornographic in nature. If bugs are encountered in these types of sites, screenshots should either not be included in the bug reports, or have the Not-Safe-For-Work portions blurred out before uploading.

Locales

We will start with the following locales, the top 50 pages (global) from Alexa. Each locale should link to a query to bring up reported bugs, a link to the tested URLs, and have a summary once it has been tested.

Global top 50 sites

  1. Google.com
  2. Youtube.com
  3. Facebook.com
  4. Baidu.com
  5. Wikipedia.org
  6. Qq.com
  7. Yahoo.com
  8. Amazon.com
  9. Taobao.com
  10. Tmall.com
  11. Reddit.com
  12. Sohu.com
  13. Live.com
  14. Vk.com
  15. Twitter.com
  16. Instagram.com
  17. Jd.com
  18. Sina.com.cn
  19. Yandex.ru
  20. Weibo.com
  21. Login.tmall.com - error 404
  22. 360.cn
  23. Blogspot.com
  24. Netflix.com
  25. Csdn.net
  26. Bilibili.com
  27. Pornhub.com
  28. Linkedin.com
  29. Twitch.tv
  30. Bing.com
  31. Yahoo.co.jp
  32. Livejasmin.com
  33. Alipay.com
  34. Mail.ru
  35. Microsoft.com
  36. Aliexpress.com
  37. Naver.com
  38. Pages.tmall.com (error)
  39. Ebay.com
  40. Whatsapp.com
  41. Microsoftonline.com (error)
  42. Tribunnews.com
  43. Okezone.com
  44. Google.co.in
  45. Office.com
  46. Amazon.co.jp
  47. Google.com.hk
  48. Xvideos.com
  49. Hao123.com
  50. Stackoverflow.com

Besides the top global 50 sites, will test the next 10 Top Sites from Alexa's country categories.

US

  1. Chase.com
  2. Craigslist.org
  3. Imgur.com
  4. Paypal.com
  5. Force.com
  6. Espn.com
  7. Cnn.com
  8. Fandom.com
  9. Imdb.com
  10. Walmart.com

UK

  1. Amazon.co.uk
  2. Google.co.uk
  3. Bbc.co.uk
  4. Ebay.co.uk
  5. Ladbible.com
  6. Theguardian.com
  7. Www.gov.uk
  8. Daily.co.uk
  9. Xhamster.com
  10. Itv.com

Germany

  1. Google.de
  2. Amazon.de
  3. Ebay.de
  4. Ebay-kleinanzeigen.de
  5. Web.de
  6. Gmx.net
  7. Ok.ru
  8. T-online.de
  9. Spiegel.de
  10. Bild.de

France

  1. Google.fr
  2. Amazon.fr
  3. Leboncoin.fr
  4. Orange.fr
  5. Free.fr
  6. Cdiscount.com
  7. Labanquepostale.fr
  8. Ebay.fr
  9. Lefigaro.fr
  10. Pole-emploi.fr

India

  1. Google.co.in
  2. Amazon.in
  3. Hotstar.com
  4. Onlinesbi.com
  5. Flipkart.com
  6. Cricbuzz.com
  7. Indiatimes.com
  8. Sarkariresult.com
  9. Hdfcbank.com
  10. Irctc.co.in

China

  1. Google.com.hk
  2. 163.com (error)
  3. Soso.com
  4. Zhihu.com
  5. Detail.tmall.com (error 404)
  6. Cnblogs.com
  7. So.com
  8. Xinhuanet.com
  9. Google.cn
  10. Douban.com

Hong Kong

  1. Discuss.com.hk
  2. Pixnet.net
  3. Bastillepost.com
  4. Aastocks.com
  5. Hkgolden.com
  6. Etnet.com.hk
  7. Appledaily.com
  8. Lihkg.com
  9. Price.com.hk
  10. Dcfever.com

Russia

  1. Avito.ru
  2. Google.ru
  3. Gismeteo.ru
  4. Kinopoisk.ru
  5. Sberbank.ru
  6. Rambler.ru
  7. Gosuslugi.ru
  8. Drom.ru
  9. Userapi.com
  10. Wildberries.ru

Results

Top Sites testing results:

  • New issues found
    • 4920 - [Bug] Leaving https://m.jd.com/ in background freezes the page - ✅ Not reproducible, closed
    • 4921 - [Bug] Selecting an item on m.aliexpres.com is not working - ✅ Verified
    • 4922 - [Bug] Cannot sign in on ebay.com via Google account or Facebook - ✅ Not reproducible, closed
    • 4939 - [Bug] Sign-in with social media buttons don't work on avito.ru - ✅ Not reproducible, closed
    • 4941 - [Bug] Some buttons from yahoo.com don't work - ✅ Not reproducible, closed
    • 4943 - [Bug] Cannot share articles from some news sites via social media icons - ✅ Verified
    • 4945 - [Bug] Tapping on an article on espn.com reloads the page ignoring the selected article - ✅ Verified
    • 4984 - [Bug] Cannot share link from reddit.com - Moved to TP list. Closed
    • 4938 - [Bug] Button "Sales of used items" doesn't work on m.price.com.hk - 02.03.2020 still reproducible
    • 5009 - [Bug] Videos don't play on itv.com - Not reproducible, closed
  • New issues 07.10.2019
    • 5812 - [Bug] The thumbnails from twitch.tv change when scrolling - Not reproducible, closed
  • New issue found 21.01.2020
    • 7809 - [Bug] Unknown protocol page displayed when accessing an article from zhihu.com - Not reproducible, closed