Auto-tools/Projects/OrangeFactor/Statistics

War on Orange: Statistics

Are oranges getting better? Worse? How do we tell?

The oranges are very tricky and don't like to be measured. They are intermittent! They like to avoid detection and will shy away from getting your hand on them. The data is noisy: between one push and the next there is no necessary correlation between either the number nor the identity of the oranges being observed.

Fortunately, we have in our arsenal mathematical tools that may be used to coerce truth from the wily oranges.

Types of Transforms

Given a time-series of data...

• filters: transform the series of data giving back the same number of points (as defined for this purpose)

• reductions: give back a scalar value, such as a mean, median, or standard deviation

• windows: take a subset of the window for further analysis

Note that a hg push series is a time series

We should move towards an architecture where an arbitrary set of filters may be applied. So you could e.g. filter, filter, window, reduce.

Filters

• smoothing

Smoothing

Gaussian smoothing:

Interesting Statistics

What are we really measuring? What does it mean? In order to combat the oranges, we must understand them. This means having in our arsenal creative measured quantities that tell a story.

Statistics should be identified that accurately and effectively convey trends in the data:

• oranges/push as a function of time (orange factor)
• most common failures as a function of time (topfails)
• for a given window, breakdown of oranges by bug number
  • in other words, are there a lot of different orange bugs?
  • is any of them a big chunk?
  • or are they highly scattered?
• push rate (number of pushes as a function of time; pushes/week, etc)
• occurance rate of orange bugs (per push) as a function of time

It is to be noted that these statistics are abstractions useful to tell a story. Fundamentally, there are discrete push events. These push events yield a certain set of oranges which are then starred.

Most Common Failures (topfails)

What particular orange bugs are most commonly seen? If I look at the last week of data, how would the oranges break down? Given the number of pushes, say, in the last week, how would these break down? Would there be a few orange bugs that, if eliminated, would significantly change the orange factor? Or are the oranges so scattered that it is unlikely that fixing any given orange would change the picture much? How does this differ from last week?