B2G/QA/Meetings/2015-10-21

From MozillaWiki
< B2G‎ | QA‎ | Meetings
Jump to: navigation, search

// For 2015/10/21

Overview

Announcements:

  • List your ideas for Topics for Moz Orlando work week : qa@mozlando
    • same idea as what we did for work week for Whistler
    • Al has submit the meeting to google submit on Wednesday
    • Next milestone: Oct. 30th - Finalize the meeting topics for FxOS teams

Off plan

Gerry - Oct. 7~23, Oct. 30~Nov. 6 (Euro Star) Mike+Shako+Walter+Shing - Oct. 30~Nov. 6 (Euro Star) marcia traveling 10-21 and 10-27

2.2r

  • Contact Window: Shing Lyu
    • Status: Shing and Al starts to reach some QA outsource partners

// Following items are for read-only

2.5 Feature

  • Bugzilla lite: Cynthia/Gerry
    • Marigold executed the testing. (No new bug)
    • Bug list: http://mzl.la/1SmOoEQ (51 bugs) - none of them are blockers
      • Done
  • Telemetry: Iris/Gerry
    • Test plan draft: https://goo.gl/6ANJuO
    • Writing test cases
    • Executing test cases in progress
    • Bug list: https://goo.gl/GXRnIH
    • User stories: code have landed, still have some issues for Metrics, so opened bug 1215654 to follow up
      • bug 1181295 - FTE UX changes for privacy, landed
      • bug 1208205 - Settings changes for privacy, landed
      • bug 1177099 - compression, landed
    • Follow up bug:
      • bug 1215654 - Metrics may not be recorded when level set to enhanced, developer marked blocking-b2g 2.5?

Bug 1188026 - Not displaying icons for photos.google.com (review?) Bug 1213894 - Progress bar doesn't appear for pinned chrome (review?) Bug 1216394 - [Browser] After you cleared browsing history, cookie, and stored data, there is no 'History' and 'Top sites' any more. (review?) Bug 1216415 - [pin the web]Fail to unpin the pinned site. (review?)

  • Private Browsing: Alison
  • TV - Bookmark Sync: Cynthia
    • Planned landing next Friday
    • I will run the testing next Monday

[PY] - Had meeting with Mars, clord, gmarty,... today. Meeting notes here -> https://docs.google.com/document/d/1uHZHZqYeRJmt8lj7qgVU6gTbiu_n11MiNNsxE6_qEHw/edit Called out as a risk.

  • Late Customization: Teri
    • Bug 1180707 - (2.5_LateCustomisation) late customisation
    • Code landed
    • Dry run to ensure flow
    • Test plan done
    • Test cases done
    • Start testing

[PY] - Suggestion from Wilfred on how to test this: Pasting his comments as is. Pls evaluate. [10/14/15, 2:42:26 AM] Wilfred Mathanaraj: use a operator like Orange and we as mozilla setup a test shelf - we then use SIMs from that country (e.g. France) from all operators in France to test the different scenarios [10/14/15, 2:42:52 AM] Wilfred Mathanaraj: the other example would be we create a vodafone france shelf - test it out with other sims [10/14/15, 2:43:07 AM] Wilfred Mathanaraj: i dont think we need operator involvement for us to test the late customisation shelf idea 


   *** Test report: https://moztrap.mozilla.org/results/cases/?filter-run=8128
    • Marigold will start second run and ad hoc test today
   *** Test run: https://moztrap.mozilla.org/runtests/run/8185/env/347/
  • Performance: Mike
    • App/Device startup time
   *** Music NGA causes performance regression: https://bugzilla.mozilla.org/show_bug.cgi?id=1210440
   *** One regression on some apps from 10/20: https://bugzilla.mozilla.org/show_bug.cgi?id=1216403
    • Media performance not landed yet
   *** https://bugzilla.mozilla.org/show_bug.cgi?id=1159343
    • Display/Graphic performance landed and Marigold will start regression test today
   *** Test run: https://moztrap.mozilla.org/runtests/run/8177/env/347/

   *** v2.5 scope re-defined and only partial landed

_* NGA: Gerry/Johan

    • [jlorenzo] Next weekly update in a few hours. Devs are still on perf improvement. The perf have been stable for the last 12 days. Visually loaded time:
      • Contacts: ± 1.0 s
      • SMS: ± 1.6s
      • Music: ± 1.3 s


  • RTL: William
   • Development progress:
       - Completed:

Dialer, FTU, System, Homescreen, Browser, Emergency Call 

        - Delayed:

Contacts, SMS, Settings(Backouted due to try fail) 

        - On-going:

Email, Calendar 

        - Implementation order: http://bit.ly/1MsORSC
   • Test progress:
       - Marigold has completed System, FTU, and Homescreen test runs.
           ~ 63.64% pass rate and 27.27% fail rate
           ~  Bug list: https://goo.gl/KGvwvo  
   • Test case creation:
           ~ 100% RTL test cases have been updated
   • What’s next?
       - Marigold is doing double check RTL test cases and add more RTL test cases for 2.5 new features.
       - Marigold is re-testing system app because they missed some components
       - Now, Marigold is executing Dialer and emergency call RTL test runs. Marigold will complete it this week. (Due date: 10/23)
       - [RISK] Many RTL patches are not yet landed.

[PY] - Other than System, have we completed testing on homescreen, settings, ftu? Are dev still waiting on gaia-header issue? [William] Yes, Phoxygen has solved gaia-header problem.

  • updates: FOTA : nhirata
    • Aries FOTA
      • new FOTA build made due to bug 1217035; testing to be done tomorrow
      • blocked by needing location for FOTA build; Aries FOTA : bug 1215667
      • client sending IMEI patch works testing still
      • blocked by needing changes from releng to make it work: Bug 1201556, Bug 1201540, bug 1211973
    • Flame FOTA : gecko/gaia only
      • fixes made, testing done, balrog changes set; Flame FOTA is now live as of 2 minutes ago.
      • caveats :
        • bug 1214737: if they don’t have the v4 base build flashed on their phones they end up on the recovery
        • bug 1215733: if the device doesn’t have the right date/time then they run into an update error. Chances of that happening are low, unless they do a fresh flash of v4
        • pvtbuild server is going away, this is a temp solution; we need a long term solution.
  • Vaani : nhirata -
    • dropped for 2.5; work still being done, QA still needed. Will be an ongoing tweak of fuzzy logic
    • please use the MozSpeech app in dogfood-latest build
    • automation talks with Johan and Andre. Waiting for Andre to make some changes to Vaani/API for reading in a recorded file.
  • Others
    • View Source: No-Jun
      • All major features are verified, there is only one bug that's being checked right now, but it's not a blocker (Bug 1204011)
      • Test suite is in moztrap

2.5 Spark/Aries-KK Smoketests

  • Tester: QAnalysts (KevinT)
    • Health Status: Aries: Green No blockers

No blockers

Severe Issues: - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1208808 Discrepancy between toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen" - https://bugzil.la/1212783 [Aries KK][Camera] The screen will be locked automatically after timeout, when you are recording.

2.5 Aries Dogfood

  • Health Status: Aries: Green. No blockers

No blockers

Severe Issues: - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1208808 Discrepancy between toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen" - https://bugzil.la/1212783 [Aries KK][Camera] The screen will be locked automatically after timeout, when you are recording.

2.5 Flame FOTA

  • Health Status: Flame: Green. No blockers

No Blockers

Issues Encountered: - https://bugzil.la/1214737 Device should check to make sure that the proper fingerprint for a FOTA is being used before trying to FOTA - https://bugzil.la/1215733 When trying to FOTA the flame device, the device gives a "status text: Update XML file malformed (200)" - https://bugzil.la/1183033 [B2G] Keyboard doesn't have click sound - https://bugzil.la/1208808 Discrepancy between toLocaleFormat and toLocaleString when timezone is unresolved - https://bugzil.la/1211266 Occasional missing icons after flashing homescreen - https://bugzil.la/1213233 [Homescreen] All icons on homescreen can't be shown, after user selected "New Home Screen" or "Default Home Screen" - https://bugzil.la/1217220 Received videos through MMS do not play and a serious error occurs when trying to watch them in video app

2.5 Exploratory

  • Tester: QAnalysts (KevinT)

=== 2.5 fullrun === (William, update: 10/21)

=== 2.5 smoketest (Marigold) === (William, update: 10/21)

  • Aries

Bug 1214280 - Users are unable to import contacts from memory card in FTU/FTE

  • Flame
   1. Bug 1188992 - [OTA] Latest Flame build is a day behind
   2. Bug 1213762 - [Flame KK][Browser]When user plays a video in YouTube site in Full Screen mode, if it switches between portrait and landscape mode, a half-screen blank area will flash past.
   3. Bug 1214280 - Users are unable to import contacts from memory card in FTU/FTE


2.5 MTBF

flame-kk 512MB avg. running hours: 23.63 Blocker: 

- https://bugzilla.mozilla.org/show_bug.cgi?id=1206634
- https://bugzilla.mozilla.org/show_bug.cgi?id=1198950

aries-kk 2GB running hours: 42.47 mtbf bugs in Q4 https://bugzilla.mozilla.org/show_bug.cgi?id=1182954

    • RA exit criteria: 100 hrs
    • CC exit criteria: 200 hrs

Previous action items

(Update: Oct. 7th)

  • -DONE [marcia] Action item to follow up with Michael Treese in Orlando
  • [team] Finalize sessions/Items for Agenda for Orlando

https://docs.google.com/document/d/1B3zIDd2s3-kqMRqzefNxDNQ8aQGs3jTo9u0XskobNc4/edit

  • [Al/Pallavi] To follow up the QA ww between Moz ww
  • [In progress] marcia: follow up with pallavi regarding community budget.


Agenda

[nhirata]: update 10/20

  • 2.5 Branching decision has been made; we will have to support 2 different branches
    • we need to figure out how to divy the work
  • pvtbuild server may be going away as early as early november
    • we need to figure out all bugs that block this. Is there a tag we should use? nhirata to organize all the bugs together for taskcluster team : QA whiteboard : [taskcluster-blocker]
    • How to perform regression windows?
      • mozregression tools
      • [jlorenzo - update after the meeting] One way we could do is:

you get the HG (not git) commit from mozilla-central or inbound you go to https://tools.taskcluster.net/index/artifacts/#gecko.v1.mozilla-central.revision.linux/gecko.v1.mozilla-central.revision.linux You do a ctrl+f on the gecko revision you

        • [JMercado] - QAnalysts does something similar using the gecko hash from the sources.xml currently found in the pvt builds. Even that is quite slow and getting the commits from hg was slower. It would also be nice to somehow find builds by gaia commit if date/time indexing is a problem.
  • nexus builds can be publically available given the right base build. Talking about details with Elvin about builds

2.5 nexus-4 kk and nexus-5-l

[atsai]: update 10/21

  • move google doc to etherpad.
    • Please be reminded for partner confidential topics, please use qa-b2g-internal mail thread to sync up/discuss.


(update October 14))

https://gaia-l10n.allizom.org/integration/gaia-central/rev/e96ce12ed450 (Not shown in the diff, but you don't have anything to do about it)

  • Foxfooding [marcia]
    • Are people in the team actively foxfooding?
    • If so, can you help answer questions that are sent to the mailing list?


- Firefox Sync strings will likely be covered by automation, but needs to be confirmed as it seems not everything is covered

  • [Jamie_] Test day info, also a few questions.


  • Scope of this meeting: Is it supposed to be public or private?
    • If private: Remove this event from the public QA calendar, and remove the notes the public wiki
    • If public: Use a public etherpad for the meeting notes

Community Updates [marcia-read only]

[Jamie_]

  • will be starting on revamp of firefox OS QA pages soon, have to talk to some people about it.


This Week - Decisions & Action Items

  • [marcia] remove Pin the Web from Test Day



Testing information

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Meetings/2015-10-21&oldid=1101787"