B2G/QA/Tips And Tricks Part2

From MozillaWiki
< B2G‎ | QA
Jump to: navigation, search

Note: special thanks to PBylenga and other QAnalydocs members for some of this information.

General Manual Testing

General

  • create a new profile
    • you can even create a new user account (ie flash the device, adb shell rm -r /data/mozilla/b2g ; reboot device); this makes sure that a new profile initialization works correctly. Usually no initialization could cause crashing
  • repeating a tests while the device is running multiple times
    • sometimes bit shifting could cause memory corruption; doing this through automation and checking the results to see if you're getting the same thing makes more sense to do. Not really recommended through manual as it becomes tedious... but you can.
  • Use a debug build of some sort (like engineering)
    • do adb logcat while testing. Sometimes you can find asserts or crasher warnings pop up in the logcat.
    • you may need to turn on some other debug prefs.

Graphics

General

  • rotate the phone
    • Sometimes changing the dimensions can cause weird graphic issues
  • Edge Gesture
    • Having the phone try to open a different app during a process can cause menus or animations to break Ex: while phone is rotating, calling keyboard, importing contacts etc.

Touch Events

  • double tap
    • just like in Zombieland, that extra tap just much help do the trick.
  • long tap
    • context menus should pop up... and then there's times when they should not.
  • Quick tap
    • Try to open menus or toggle things on/off during the animation of opening something else
  • Hardware interupption
    • After tapping or selecting something, tap the homebutton or power button. As an added bonus, try double tapping the home or power buttons during this time

Top Sites

When Testing Top Sites it’s important explore specific areas are functioning.

When testing websites we want to ensure user does not have a degraded experience by using a Firefox OS device rather than something else.

Scroll Events

  • Depending on the webpage check to see if you can do a horizantal scroll; some allow for horizontal scroll or vertical scroll; some don’t allow for zoom. You will need to check the web page source. This also depends on the UA. See UA section down below. Mobile v Website version of the page.
    • Scrolling while content loads
    • Scrolling large amounts of text/images check for black or white checkerboxing, stutters
    • Scroll over embedded videos
    • Scroll over an embedded section that also is scrollable (etherpad)
    • Zoom: ( depends on the site, some mobile sites do not allow you to zoom, you will have to check the header in the website code )
  • check to see how text, images, videos are handled when zoomed in/out and also pan around them
  • pinch zoom
      • in
      • out
      • double tap zoom
      • Overscroll ( ie try to scroll past the top or past the bottom of the page )

Menus

  • Tap a menu drop down, ensure it opens and stays open
  • check menu scrollability
  • rotate orientation with Menu open
  • attempt to open menu’s while page is loading

Share events

  • Add page to homescreen
  • Bookmark page
  • Edit bookmark
  • Email links by long pressing
  • share videos
  • attempt “Mail to:” email links

Flyouts, popups, overlays

  • Checkout cart UI
  • Information flyout boxes
  • Forms
  • Logins
  • popups

Media:

Image:

  • long tap on an image and save
  • try to get to various different media types ( animated gifs, jpg, png ) if available
  • Video
  • long tap on a video and save
  • play different video types if there are those

Streaming:

  • Audio conflict with Phone apps (Music, video, gallery, dialer)
  • stop, start

Fields:

  • Depending on the field the correct keyboard type pops up, typing is allowed
  • Check text descenders for characters: jpqyg

Links:

Cookie storage/Data Store:

  • login/password saves

Device Rotation:

  • with any of the above rotate the device from landscape to portrait or vice versa

User Agent:

  • we have a general user agent that can be found by going to :
    • "whatsmyuseragent.com"
  • we have 2 override files ua-update.json found in /system/b2g , /data/local
  • Doing a comparison by setting the general UA on the desktop browser and using this addon :

Geolocation:

  • if a page has geolocation ability ( ie find the closest store to me ) option use that

Exploratory Testing

  • No right or wrong... perhaps "better" as in more efficient or results are more satisfying... or ZUL. I forget which.

Be curious: Ask How/Why questions alot:

  • Why does the feature work the way it does?
  • How does the feature work the way it does?
  • How many ways into the feature are there?
  • How many ways out of a feature are there?

Why : Chances are a dev or triage team will ask you the question in a bug of what happens if you go through a different way to get to the same feature... Short cut by testing and placing in notes what does and doesn't work; and provide as much info so the dev could try to find a fix in a faster amount of time. Example of Dev asking for more details : https://wiki.mozilla.org/Gaia/Email/RequiredBugInfo

Bugs occur when :

  • There's an oversight
    • other people coding
    • forgetting a case
  • There's a mistake
    • simple syntax
    • logic flow, etc.

Get to understanding how things work

  • Understanding the architecture, how things are built, how they are built... what they are capable of, what the limitations are... what design pattern they used in order to build... the more you understand and understand the context that the dev built it helps you find bugs faster/better. Better in that the dev will understand how to fix it faster. Being able to talk the same lingo helps out a lot as well.

Model View Controller (MVC) : ( http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller )

  • This is how I test, even though in most cases, the software itself isn't necessarily designed this way
  • Reason: It's easier to conceptualize
    • UI
      • l10n bugs
      • graphic icons
      • glitches
        • accessibility issues
      • Data
        • special edge cases such as 0, 1, null, etc
      • Controller
        • Business logic (ie functionality issues)
          • User Flow logic (ie Usability issues)

Be aware of States :

  • In comp sci: I was taught to comment pre, post, and expected results in a function to comment what a function does. It's quite applicable in QA.
  • in a micro sense we could potentially figure out various states of data going into a function and going out
  • if you plot out what things you are doing and testing, and remember what you are doing in detail (ie what buttons, ui you touched) then chances are steps to reproduce (STR) are going to be easy to figure out.
  • moving outwards, it still applies to functionality of a feature:

example :

    • music app is turned on while dialer is running
    • if the music app being turned on is a state different from nothing else running and you could potentially run into a bug.
  • outlines help
  • setting up test cases to a certain order helps
  • caveat: test in various ways... don't do the same thing the same way each time.

Check the checked in code to see where to test:

  • “Insanity: doing the same thing over and over again and expecting different results.” - Albert Einstein
  • If nothing had changed in that part of the code, and the code doesn't affect that area, and you have already tested that area... do you really expect to find a bug (easily)?
  • looking at where the change takes place is most likely the easiest and fastest way to find bugs... and checking "all" ways of impact:

Example : bug 820147 ( https://bugzilla.mozilla.org/show_bug.cgi?id=820147 ) ; bug 813275 ( https://bugzilla.mozilla.org/show_bug.cgi?id=813275 )

(you guys don't have to worry so much .. ) UXbranch: https://github.com/gordonbrander/gaia

Be curious about available tools to help make your job easier and you get done faster.

  • if you ask the question : why isn't there some sort of tool to do x task?
    • you may want to check to see if there already is one.

Automate if possible, esp test cases. Once a test case is run, it turns into regression testing. Granted there has been a lot of regression bugs occuring in this project (due to back end change and front end ui not accounting for that change) and things of that nature... generally speaking regression bugs should be a lot fewer in most other projects that don't have back end changes happening too often. The time it takes to do regression testing is time that could be done doing exploratory testing and uncovering new ground/more bugs. and, if you'd like something automated, please write it up clearly and post it as a GitHub Issue here: https://github.com/mozilla/gaia-ui-tests/issues?labels=gaia+smoketest&state=open

  • If you can't automate, well... that sucks. do it manually.

Tools

Don't forget to use the tools that B2G comes with for better debugging! https://developer.mozilla.org/en-US/Firefox_OS/Debugging/Developer_settings

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Tips_And_Tricks_Part2&oldid=1053464"