B2G/QA/Test Automation/mozcingi

From MozillaWiki
< B2G‎ | QA‎ | Test Automation
Jump to: navigation, search

About The Project

This project implements the fuzzing testing framework against B2G devices.

Cingi: vocabulary from Taiwan aboriginal "Yami", which means spy on sth.

Fuzz testing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems. It is a form of random testing which has been used for testing hardware or software.

Current Feature

  • Able to run multiple instance on different machine concurrently
  • Support real-time configuration change
  • Able to support randomize header tag on audio file
  • Able to support running on gip

Support list

  • Real device - Aries
  • MozITP - real device - Aries
  • MozITP - mullet - phone

Current Environment & Setup

Fuzzing testing tests are run by automation; the test suite collects specific test cases on certain target for example:audio play, etc., mutate the testing data and inject to those test cases. Right now we have one 1 fuzzer which is focus on audio file header mutation.

How often are the tests run?

Fuzzying testing is purposed to test on any version or branch. It will run 7x24 hrs until finding a crash on target.

Where can I see the results?

mozCingi currently integrate with FuzzManager, once the test case execution is finished, it will upload the data to FuzzManager. You can see the detail information there.

Running the tests

Let's go through the steps required to set up the mozCingi test environment and run the tests on your local machine and devices.

Prerequisites 

   Ubuntu 12.04 (or better) x64 
   A Firefox OS device ALREADY FLASHED with an ENGINEERING build of Firefox OS 2.5
   ADB installed, and the environment variable ADB_PATH pointing to your ADB location.

If you are on Ubuntu, you need to check that it is configured to support the USB connection to the device. To verify this, connect the device to your computer via USB, open a terminal and enter the adb logcat command to see if it connects. If not, you may need to set up a udev rule for the device.

Step 1: Clone the mozCingi repository

Clone the mozcingi repository in a folder you will remember: 

git clone https://github.com/ShakoHo/mozCingi

Step 2: Run the setup

Install all requirements including mozITP and FuzzManager 

make mutagen-install

Step 3: Set configuration variable

You can leverage the sample configuration under conf/, copy the sample configuration and modify it 

cp conf/mutagenFuzzer.json.bak conf/mutagenFuzzer.json

vi conf/mutagenFuzzer.json

Step 4: Switch to python virtual environment

switch to python virtual environment, just created by make mutagen-install 

source ./env/bin/activate

Step 5: Run the tests

use cingi command to start the whole framework, you could add --dirpath to specify configuration folder 

cingi --dirpath conf

Contributing to the project

If you have any questions about the mozCingi or are interested in contributing to this important automation development effort, feel free to contact us at sho@mozilla.com

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/Test_Automation/mozcingi&oldid=1114874"