canmove, Confirmed users
2,041
edits
B2G/QA/2014-10-02 Performance Acceptance: Difference between revisions
Jump to navigation
Jump to search
B2G/QA/2014-10-02 Performance Acceptance (view source)
Revision as of 23:39, 6 October 2014
, 6 October 2014→Result Analysis
| Line 44: | Line 44: | ||
;Median: 50% of launches are faster than this. This can be considered typical performance, but it's important to note that 50% of launches are slower than this, and they could be much slower. The shape of the distribution is important. | ;Median: 50% of launches are faster than this. This can be considered typical performance, but it's important to note that 50% of launches are slower than this, and they could be much slower. The shape of the distribution is important. | ||
;95th Percentile: 95% of launches are faster than this. This is more quality-oriented statistic commonly used for page load and other task-time measurements. It is not dependent on the shape of the distribution and better represents a performance guarantee. | ;95th Percentile (p95): 95% of launches are faster than this. This is more quality-oriented statistic commonly used for page load and other task-time measurements. It is not dependent on the shape of the distribution and better represents a performance guarantee. | ||
Distributions for launch times are positive-skewed asymmetric, rather than normal. This is typical of load-time and other task-time tests where a hard lower-bound to completion time applies. Therefore, other statistics that apply to normal distributions such as mean, standard deviation, confidence intervals, etc., are potentially misleading and are not reported here. They are available in the summary data sets, but their validity is questionable. | Distributions for launch times are positive-skewed asymmetric, rather than normal. This is typical of load-time and other task-time tests where a hard lower-bound to completion time applies. Therefore, other statistics that apply to normal distributions such as mean, standard deviation, confidence intervals, etc., are potentially misleading and are not reported here. They are available in the summary data sets, but their validity is questionable. | ||