Performance/Status Meetings/2007-July-18

« Back to Status Meetings

Participants

zach, sdwilsh, mwu, shebs, joduinn, vlad, dbaron, bzbarsky, bhearsum

Infrastructure

• Generate reliable, relevant performance data (already underway as talos). Talos status update? http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest
  • running against hourly builds on trunk, see MozillaTest
  • lots of crashes with historical runs, wondering if its doable.
  • released branch points already available on 1.8 branch.
  • robcee/alice to start using smaller Tp set

• Discuss talos cpu usage metric
  • cpu usage numbers... discussion on how cpu usage being calculated.
  • going to ignore those numbers for now, no humans looking at them.
  • AI:zach will merge changes

• Areas where help is needed
• expand the scope of performance testing beyond Ts/Tp/TXUL/TDHMTL

• reduce noise in tests
  • reduce to ~1% (suggested by bz, not started)
  • move perf tests to chrome, so we get more reliable results, and can test more than just content
  • AI:vlad & dbaron to checkin hopefully today

• improve performance reporting and analyses:
  • Better reports for sheriffs to easily spot perf regressions
  • Tracking down specific performance issues
• stats change to track AUS usage by osversion. bug#382044. AI:justin

• Priorities for infra:
  • Generate historical baselines
  • General profile data regularly on builds
  • Getting the perf numbers more stable
  • Developing the graph server to display time spent in each module

• New ideas
  • Question: How are we tracking perf bugs, specifically, and are we doing this the same way we are triaging security bugs? Can we do it the same way if not? (AI:damon)

Gecko: Perf discussion

• Perfathon update
  • vlad has data

• pageload test rewrite (bug 387110)
  • since we have a chance at a clean break, we should look at how the composite scores are calculated
  • do we still want to ignore the biggest value when computing the mean and stddev? (there is caching and stuff that happens during the first time a page is loaded that ends up beneficially effecting subsequent page loads)
  • do we want to more cycle more intelligently?
    • e.g. do we want to keep cycling (up to some max) until the stddev drops below some number?
    • do we want to try to intelligently throw outliers out?

• update on timer resolution on windows from Rob Arnold.
  • Done, checked in earlier this week.

• Talos results
  • What's it look like for 1.8 -> 1.9?

• Todo: verification of test results
  • Fonts, valid rendering, etc.
  • TODO: Stuart: verify that concerns about talos results are addressed

• probe update
  • stan taking over mozilla/gecko side of probe work
    • will get Sun's dtrace patch as well as xp-probes checked in to new toplevel dir, then work on merging the two
  • crowder taking spidermonkey side
    • will be dtrace only
  • module owners should let stan know if they have any situations that they'd like to analyze, so that he has more information about how to structure the probe work.
    • for example, examining loading from network performance vs. loading from disk
  • Sun guys will be here on Aug 30 to do a presentation on dtrace, followed by hands-on work with a smaller group of engineers

Previous weeks

• jrpof tinderbox

• synthetic tests (TODO)
  • put specific tests into talos framework
  • poach mochitest performance tests
  • individual reftest-like items

• need to get a bunch of people looking at profiles (TODO)
  • take up some of these meetings to sit down and look at profiles
  • timer-based profiling is better (vtune/jprof/oprofile/etc., not quantify)
  • TODO: vlad to generate profile for next week's meeting

• running without Fx chrome
  • new pageloader stuff can do this

• examining default theme for performance issues

• Tp vs. Tp2
  • new pageloader stuff avoids this problem
  • Tp2 was flawed in some ways -- one big one is that it loaded pages in an iframe, so initial paint delay was never coming into play, and we were potentially reflowing more thannecessary

• what knobs/perfs do we actually have?
  • what happens when we change these perfs?

Other Information

Related Bugs