Hasal: Difference between revisions
Jump to navigation
Jump to search
Full Query
(update hasal current status) |
(update hasal current status) |
||
| Line 2: | Line 2: | ||
== State of this project == | == State of this project == | ||
We are currently in the middle of phase 2. Execution summary as below: | |||
=== Execution Summary === | |||
* Script Complete: test script is implemented and tested | |||
* Execution Complete: test script run over 30 times and running time is recorded | |||
* Profiler Data Complete: Gecko profiler, har profiler and performance timing object upload to bugzilla | |||
* Analyze Data Complete: Analyze the gecko profiler data and find the most running time of different time unit (between navigationStart and domLoading) | |||
{| class="wikitable" | |||
|- | |||
! Priority !! Script Complete !! Execution Complete !! Profiler Data Complete !! Analyze Data Complete | |||
|- | |||
| High || 27/27 || 27/27 || 8/27 || 8/27 | |||
|- | |||
| Medium || 0/25 || 0/25 || 0/25 || 0/25 | |||
|- | |||
| Low || 0/14 || 0/14 || 0/14 || 0/14 | |||
|} | |||
== About this project == | == About this project == | ||
| Line 12: | Line 27: | ||
== Project Plan == | == Project Plan == | ||
** [https://goo.gl/umtHvq Resource Plan] | ** [https://goo.gl/umtHvq Resource Plan] | ||
** [https://goo.gl/n4TnoL Test Plan] | ** [https://goo.gl/n4TnoL Test Plan] | ||
| Line 17: | Line 33: | ||
=== Google Doc === | === Google Doc === | ||
==== Phase 1 (Done) ==== | ==== Phase 1 (Done) ==== | ||
* Build up a workable framework that can run some manual created test cases | * Build up a workable framework that can run some manual created test cases | ||
** Trigger actions on browser | ** Trigger actions on browser | ||
| Line 27: | Line 45: | ||
==== Phase 2 ==== | ==== Phase 2 ==== | ||
* Create test cases based on test matrices | * Create test cases based on test matrices | ||
* Create test scripts for high priority cases and execute them | * Create test scripts for high priority cases and execute them | ||
* Generate profiler data for each testing result | * Generate profiler data for each testing result | ||
* Find out the module and method which cost the most running time | * Find out the module and method which cost the most running time | ||
* Go through the whole process from analyzing to provide information | * Go through the whole process from analyzing to provide information | ||
| Line 41: | Line 57: | ||
==== Phase 3 ==== | ==== Phase 3 ==== | ||
* Create scripts for medium and low priority cases and execute them | * Create scripts for medium and low priority cases and execute them | ||
* Find out the module and method which cost the most running time | * Find out the module and method which cost the most running time | ||
== Team == | == Team == | ||
Revision as of 07:15, 16 May 2016
A performance measurement project for Google Doc, Google Sheet, and Google Slide.
State of this project
We are currently in the middle of phase 2. Execution summary as below:
Execution Summary
- Script Complete: test script is implemented and tested
- Execution Complete: test script run over 30 times and running time is recorded
- Profiler Data Complete: Gecko profiler, har profiler and performance timing object upload to bugzilla
- Analyze Data Complete: Analyze the gecko profiler data and find the most running time of different time unit (between navigationStart and domLoading)
| Priority | Script Complete | Execution Complete | Profiler Data Complete | Analyze Data Complete |
|---|---|---|---|---|
| High | 27/27 | 27/27 | 8/27 | 8/27 |
| Medium | 0/25 | 0/25 | 0/25 | 0/25 |
| Low | 0/14 | 0/14 | 0/14 | 0/14 |
About this project
We are trying to measure the difference between Firefox and Chrome on high visibility web apps (Google Doc, Google Sheet, Google Slide, etc.)
We use the video recording to record the user actions and extract the frames to get the running time from user perspective. Based on same user actions, we measure the difference between Firefox and Chrome.
Project Plan
Google Doc
Phase 1 (Done)
- Build up a workable framework that can run some manual created test cases
- Trigger actions on browser
- Record video on the background
- Enable profiler on firefox automatically
- Convert the video to image automatically
- Compare images with sample images
- Measure the difference between Firefox and Chrome for sample user actions (40% found between Firefox and Chrome)
Phase 2
- Create test cases based on test matrices
- Create test scripts for high priority cases and execute them
- Generate profiler data for each testing result
- Find out the module and method which cost the most running time
- Go through the whole process from analyzing to provide information
- Framework improvement
- Replace Selenium with Sikuli
- Get the medium value and other statistics
- Support windows and mac
Phase 3
- Create scripts for medium and low priority cases and execute them
- Find out the module and method which cost the most running time
Team
| Name | Email address | |
|---|---|---|
| Shako Ho | sho@mozilla.com | You can contact him to get more project detail. |
| Askeing Yen | askeing@mozilla.com | |
| Cynthia Tang | ctang@mozilla.com | |
| Farmer Tseng | fatseng@mozilla.com | |
| Walter Chen | wachen@mozilla.com |
Working Items
| ID | Component | Summary | Status | Assigned to | Target milestone | Depends on |
|---|---|---|---|---|---|---|
| 1278554 | Graphics: Canvas2D | CanvasRenderingContext2D::DrawImage takes lots of time when scrolling Google Spreadsheet | NEW | --- | 1237058 | |
| 1307024 | General | [Perf] [google docs] Firefox will get 3~9% faster when running gdoc test cases with chrome user agent | NEW | --- | 1321623 |
2 Total; 2 Open (100%); 0 Resolved (0%); 0 Verified (0%);
Setup environment
Testing Framework
Following the steps here [1] to setup the framework. For the time being, only Ubuntu is officially supported as a testing environment; you can use OS X but you'll have to manually solve some package dependency issue. Note that Ubuntu 12.04 and 14.04 is the best-supported testing OS.
Analysis Tool
Following the steps here [2] to setup the tool. For the time being, only Ubuntu is officially supported as a testing environment; you can use OS X but you'll have to manually solve some package dependency issue. Note that Ubuntu 12.04 and 14.04 is the best-supported testing OS.