MozillaWiki

QA SoftVision Team/Mobile/Writing tests: Difference between revisions

Page Discussion

QA SoftVision Team/Mobile/Writing tests (view source)

Revision as of 17:08, 21 December 2014

13,272 bytes added ,  21 December 2014
m
Bug 1082602
No edit summary
m (Bug 1082602)
 
(29 intermediate revisions by 2 users not shown)
Line 7: Line 7:
* The test file name should be test[Feature_to_be_tested].java.in
* The test file name should be test[Feature_to_be_tested].java.in
* The test must be included in the same package as all other Robocop tests
* The test must be included in the same package as all other Robocop tests
= How to start when writing a new test =
* Setup the enviroment. Follow the instructions [https://wiki.mozilla.org/QA_SoftVision_Team/Mobile/Robocop_automation_setup here].
* Download sources. More info can be found [https://wiki.mozilla.org/QA_SoftVision_Team/Mobile/Robocop_automation_setup#Download_sources_using_Mercurial here] about how to do it.
* If you already have the sources downloaded update to the lates version by running the command in the sources directory:
hg pull -u
* Make the build by running:
make -f client.mk
* Create a new file named "test" followed by the test name and with the extention "java.in" : for e.g. ''testBookmarks.java.in''
* Copy the basic structure of the test presented in the [https://wiki.mozilla.org/QA_SoftVision_Team/Mobile/Writing_tests#Basic_structure_of_the_test next section] to the test making sure to match the test name and the class and constructure names.
* Create a new mercurial queue patch on the repository by running in the source folder the command:
hg qnew <patch_name>
* Add the new test file to the queue by running the command:
hg add mobile/android/base/tests/<test_file_complete_name>
* Open robocop.ini from mobile/android/base/tests and add after the last Robocop test a new line with the test name between sqare brackets ( "[" and "]")
* Write the test, compile robocop ([https://wiki.mozilla.org/QA_SoftVision_Team/Mobile/Writing_tests#Updating_your_test_directory here is how]) and run the test in order to test that it works
* After the test works follow the instruction to [https://wiki.mozilla.org/QA_SoftVision_Team/Mobile/Writing_tests#Creating_a_diff_patch_file create a patch] so the test can be uploaded to a bug and integrated in mozilla.
* Before updating the sources or after the patch is completed make sure to return the sources to the unmodified version by running the command in the source folder until there are no patches in the queue:
hg qpop


= Basic structure of the test =  
= Basic structure of the test =  


  package org.mozilla.fennec_root.tests; // Robotium test package
#filter substitution
  package @ANDROID_PACKAGE_NAME@.tests;
import org.mozilla.fennec_root.*; // import the package in order to be able to reuse classes if needed
   
   
import @ANDROID_PACKAGE_NAME@.*;
 
  /*
  /*
  Insert info about the test here - a basic description of what the test does
  Insert info about the test here - a basic description of what the test does
Line 39: Line 59:
* If your tests extends BaseTest then you will not need to add the methods setUp() and tearDown() as they are already defined in BaseTest
* If your tests extends BaseTest then you will not need to add the methods setUp() and tearDown() as they are already defined in BaseTest
* If you need to do extra clean-up, like removing entries you added in a database for e.g., the tearDown() method can be overwrittern but make sure to call '''super.tearDown()''' as needed in the method. The same is applicable for setUp().
* If you need to do extra clean-up, like removing entries you added in a database for e.g., the tearDown() method can be overwrittern but make sure to call '''super.tearDown()''' as needed in the method. The same is applicable for setUp().
= APIs  =
Robotium itself provides a rich API through the Solo class - javadocs for Solo are available at [1].
"Robocop" provides an additional API to make common tasks easier. The main interfaces are Actions, Elements, and Driver.
Actions provides commonly used non-element specific actions that can be taken on the application, such as dragging and sending key events.
<pre>
Actions
  //This will cause this process to spin until the gecko fires a specific JSON event, such as DOMContentLoaded
  void waitForGeckoEvent(String geckoEvent);
  //Clicks the given Key (Actions.SpecialKey.[DOWN|UP|LEFT|RIGHT|ENTER])
  void sendSpecialKey(SpecialKey button)
  //Sends a string of characters to the system. (most have been implemented but not all)
  void sendKeys(String keysToSend);
  //Sends a drag action across the screen
  void drag(int startingX, int endingX, int startingY, int endingY)
  // Run a sql query on the specified database
  public Cursor querySql(String dbPath, String sql);
</pre>
Element represents each of the available UI objects in Fennec including the Awesomebar, the 'tabs' button, and different lists and menus.
<pre>
Element
  //To click the element.
  void click()
  //Returns true if the element is currently displayed
  boolean isDisplayed();
  //Returns the text currently displayed on the element, or direct sub-elements.
  String getText();
</pre>
Driver finds elements and provides info about the UI.
<pre>
Driver
  //This is used to find elements given their id's name.
  Element findElement(String name);
  //This is used for getting information on scrolls. NOTE: It must be used for the next three methods to return useful information
  void setupScrollHandling();
  int getPageHeight(); //The total height of the page.
  int getScrollHeight(); //How far down the screen the client has scrolled.
  int getHeight(); //The height of the client's view.
  //The following are used to give information on the graphical location of the Gecko view on the screen.
  int getGeckoTop();
  int getGeckoLeft();
  int getGeckoHeight();
  int getGeckoWidth();
</pre>
Finally, an evolving set of test base classes - BaseTest, PixelTest, etc - can be leveraged for some types of tests.
= Usable IDs =
The following is a list of ids that can be used with Driver.findElement(). Most of the following ids have not been tested, so might have unexpeced results, or require increased APIs for them. To know how a given object is used, in mobile/android/base, grep R.id.[id-name] *
(from Objdir/mobile/android/base/R.java#id)
<pre>
abouthome_content
add_tab
addons
address_bar
agent_mode
all_pages_list
awesome_bar
awesome_screen
awesomebar_button
awesomebar_tabs
awesomebar_text
background
bookmark
bookmark_icon
bookmark_title
bookmark_url
bookmarks_list
browser_toolbar
close
container
doorhanger_choices
doorhanger_container
doorhanger_title
favicon
forward
gecko_layout
grid
history_list
info
list
main_layout
notification_image
notification_progressbar
notification_text
notification_title
outline
plugin_container
preferences
quit
recommended_addon_list
reload
save_as_pdf
screenshot
select_list
share
site_security
stop
tabs
tabs_count
title
url
</pre>
= Gecko Events you can wait for =
* There are a set of messages sent from Gecko to Java to signal different actions
* Also Java can register observers to catch changes to deferent actions
* Here is a list of Gecko Events on which you can create EventExpectors and block until the message is received
<pre>
Addons:Change
Addons:All - shown when sending all addons to java as a json
Addons:FetchAll - get the state of all addons
Browser:Quit
Browser:ZoomToRect - zoom to element (viewport if the element is to large)
Browser:ZoomToPageWidth
Campaign:Set - Update the prefs for this session - related to distribution
CharEncoding:Data
CharEncoding:Get
CharEncoding:Set
CharEncoding:State
Content:LoadError
Content:PageShow
Content:ReaderEnabled
Content:StateChange - change in network state
Content:LocationChange - change of location
Content:SecurityChange
DesktopMode:Change
Distribution:Set
Distribution:Set:OK
DOMFullScreen:Start
DOMFullScreen:Stop
DOMContentLoaded
DOMLinkAdded
DOMMetaAdded
DOMModalDialogClosed
DOMUpdatePageReport
DOMTitleChanged
DOMWindowClose
DOMWillOpenModalDialog
Doorhanger:Add
Doorhanger:Remove
Doorhanger:Reply
Feedback:LastUrl
Feedback:OpenPlayStore
Feedback:MaybeLater
FilePicker:Show
FormAssist:Hide - shown when hide request given
FormAssist:AutoComplete - shown when autocomplete is displayed
FormAssist:Blocklisted
FormAssist:Hidden - shown when the form assist has been hidden
FormAssist:ValidationMessage - shown when a vaildation message is displayed
FormHistory:Init - initialize form history db on update
FullScreen:Exit
Gecko:Ready
Gesture:CancelTouch
Gesture:DoubleTap
Gesture:LongPress
Gesture:Scroll
Gesture:ScrollAck
Gesture:SingleTap
Link:Favicon
Link:Feed
Menu:Add
Menu:Clicked
Menu:Update
Menu:Remove
MozApplicationManifest
MozMagnifyGesture
MozScrolledAreaChanged
nsPref:changed
OrderedBroadcast:Send
pageshow
Panning:Override
Panning:CancelOverride
PanZoom:StateChange
Passwords:Init - initialize passwords db on update
Permissions:Data
Prompt:Show
PluginBindingAttached
Pref:Change
Preferences:Get
Preferences:Set
Preferences:Observe
Preferences:RemoveObservers
PrivateBrowsing:Data
Reader:Add
Reader:Added
Reader:FaviconRequest
Reader:GoToReadingList
Reader:Remove
Reader:Share
robocop:scroll
SaveAs:PDF
Sanitize:ClearData
Sanitize:ClearHistory
Sanitize:Finished
scroll
ScrollTo:FocusedInput
SearchEngines:Data
SearchEngines:Get
Session:Back
Session:Forward
Session:Reload
Session:RestoreEnd
Session:ShowHistory
Session:StatePurged
Session:Stop
SessionHistory
Share:Text
SharedPreferences:Set
SharedPreferences:Observe
Shortcut:Remove
Tab:Close
Tab:Closed
Tab:HasTouchListener
Tab:Load
Tab:Select
Tab:Selected
Tab:ViewportMetadata
Telemetry:Add
Telemetry:Gather
TextSelection:HideHandles
TextSelection:PositionHandles
TextSelection:ShowHandles
Toast:Show
ToggleChrome:Focus
ToggleChrome:Show
ToggleChrome:Hide
Viewport:Change
Viewport:Flush
Viewport:FixedMarginsChanged
Wallpaper:Set
WebApps:Open
WebApps:PreInstall
WebApps:PostInstall
Window:Resize
</pre>


= Basic code sequences to do different actions =
= Basic code sequences to do different actions =


== Load page ==
== Load page ==
* Note that the tests need to use '''local pages''' in order to run on the mozilla setup. Outside pages are not accessible
* Using an URL
* Using an URL
  String url = "<insert_link_here>";
  String url = "<insert_link_here>";
Line 144: Line 409:
  mAsserter.ok(list != null, "checking that all pages list exists", list.toString()); // tests if  
  mAsserter.ok(list != null, "checking that all pages list exists", list.toString()); // tests if  
  the list is null and if it isn't it prints the list after printing the message of what the test does
  the list is null and if it isn't it prints the list after printing the message of what the test does
= Debugging tests =
== See the screenshot from the fail in the try run ==
* Open the link to the tryrun and view the full logs of the failed run
* Go the the position the fail you are interested in has happened
* Search for "base64" and copy the hole line from the log
* Remove the "SCREENSHOT" from the beginning of the string and paste it in the URL bar of the browser
* You can see the screenshot of the output the moment the test failed
== Debugging using logs ==
* You can use mAsserter.dumpLog(Sting) or mAsserter.ok(true, Sting, String) to print any string or value of any variable
* Note that you will need to cast the variable to String in order to print it: String.valueOf(Variable)
= Substituting mSolo.sleep =
* Don't use sleep unless absolutely necessary
* It's ok to use sleep while creating the test but at the end try and substitute it
* Here are some alternatives:
** waitForText(String waitText)
** waitForEnabledText(String waitText)
** blockForEvent/ blockForEventData - gecko event expecters
** waitForCondition
** waitForTest
** waitForPaint
** waitForView(View view)
** waitForView(View view, int timeout, boolean scroll)
** waitForDialogToOpen()
** waitForDialogToClose()
** waitForView(int id)
** waitForView(int id, int minimumNumberOfMatches, int timeout)
** waitForView(int id, int minimumNumberOfMatches, int timeout, boolean scroll)
** waitForActivity(name)
** waitForActivity(Class<? extends Activity> activityClass)
** waitForActivity(Class<? extends Activity> activityClass, int timeout)


= Updating your test directory =
= Updating your test directory =
Line 158: Line 453:
make -C build/mobile/robocop/
make -C build/mobile/robocop/
make package </pre>
make package </pre>
= Creating a diff patch file =
All commands need to be run on the main root source folder.
Steps to perform for new patches:
* Create a new patch
* Add or modify any files
* Refresh the repository
* Pop the patch to have a clean repository
* Push the patch back
* Export the patch
* Pop the patch to clean up the repository
Steps to perform for existing patches:
* Push the patch to the repository
* Add or modify any files
* Refresh the repository
* Pop the patch to have a clean repository
* Push the patch back
* Export the patch
* Pop the patch to clean up the repository
== Create a new patch in the Mercurial Queue ==
Merurial can create patch queue (patch stack) so the user can keep his repository clean and can add or work on multiple patches on the same repository
hg pull -u
hg qnew <patch name>
You need to do hg pull to have the repository up to date in order to create a correct diff patch
== Adding new files to be tracked by Mercurial ==
You need to add new files to be tracked otherwise the diff will be created only for the existing changed files
Run the following command for each new file
hg add mobile/android/base/tests/<test_file>
== Refresh the repository ==
When first creating a patch:
hg qrefresh -m "Commit message"
When editing patches:
hg qrefresh
The commit message has the general form: "Bug <number> - <Description>". The commit message should be set for a new patch and will appear in the patch file.
== Poping the patch ==
Removes the patch from the patch stack (Queue)
hg qpop
== Pushing the patch ==
Adds the last patch to the top of the queue. Alternatively you can specify a patch name to add.
hg push -v [<patch_name>]
== Viewing the diff file ==
hg qdiff
== Exporting the patch ==
Exports the top patch in the stack to a file that can be submitted for review
hg export tip > <patch_name.patch>
== Remove files from the current patch ==
In case you added an unwanted file to the patch it can be removed by running:
hg rm -f mobile/android/base/tests/<test_file>
== See a list of all patches in the queue and changed files outside of patches ==
* If there are files displayed that means that the file has been changed outside of any patch
hg status
== Apply a certain patch/patch tree ==
* This command will apply the patch or the sequence of patches until the selected patch
hg qgoto <patch_name>
= Adding a patch to the tryserver =
* In order to add a patch to the tryserver you need to get Commit access level 1
* Follow the steps listed [http://www.mozilla.org/hacking/committer/ here] to get the access
* All patches must contain a trychooser patch on top of it in order to limit the tests only to what it is needed. Build the statement for it using [http://trychooser.pub.build.mozilla.org/ this page]
== Create the try patch over the existing patch queue ==
hg qnew -m "try: -b o -p android -u robocop-1,robocop-2 -t none" try
== Push the changes to the tryserver ==
hg push -f -rtip ssh://<user>@hg.mozilla.org/try/
* This will push the changes and will give you a Treeherder link to your push
* When the building process for the test build will start you will also receive an email with the push info
== Unlock the patch queue ==
* From Mercurial 2.0 the queue will be locked if you pushed to try
* You can unlock the queue in order to remove the patches or change them
hg phase -f --draft qbase:tip
== Removing the try patch from the queue ==
* After you are done with the push you can delete the try patch so you can add it to other patches for push
hg qremove try
Edmorley
canmove, Confirmed users
1,126

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/469345...1043340"