Performance/Status Meetings/2007-July-18

« Back to Status Meetings

Participants

zach, sdwilsh, mwu, shebs, joduinn, vlad, dbaron, bzbarsky, bhearsum

Infrastructure

• Generate reliable, relevant performance data (already underway as talos). Talos status update? http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest
  • running against hourly builds on trunk, see MozillaTest
  • lots of crashes with historical runs, wondering if its doable.
  • released branch points already available on 1.8 branch.
  • robcee/alice to start using smaller Tp set

• Discuss talos cpu usage metric
  • cpu usage numbers... discussion on how cpu usage being calculated.
  • going to ignore those numbers for now, no humans looking at them.
  • AI:zach will merge changes

• Areas where help is needed
• expand the scope of performance testing beyond Ts/Tp/TXUL/TDHMTL

• reduce noise in tests
  • reduce to ~1% (suggested by bz, not started)
  • move perf tests to chrome, so we get more reliable results, and can test more than just content
  • AI:vlad & dbaron to checkin hopefully today

• improve performance reporting and analyses:
  • Better reports for sheriffs to easily spot perf regressions
  • Tracking down specific performance issues
• stats change to track AUS usage by osversion. bug#382044. AI:justin

• Priorities for infra:
  • Generate historical baselines
  • General profile data regularly on builds
  • Getting the perf numbers more stable
  • Developing the graph server to display time spent in each module

• New ideas
  • Question: How are we tracking perf bugs, specifically, and are we doing this the same way we are triaging security bugs? Can we do it the same way if not? (AI:damon)

Gecko: Perf discussion

• Perfathon update
  • vlad has data

• pageload test rewrite (bug 387110)
  • since we have a chance at a clean break, we should look at how the composite scores are calculated
  • do we still want to ignore the biggest value when computing the mean and stddev? (there is caching and stuff that happens during the first time a page is loaded that ends up beneficially effecting subsequent page loads)
  • do we want to more cycle more intelligently?
    • e.g. do we want to keep cycling (up to some max) until the stddev drops below some number?
    • do we want to try to intelligently throw outliers out?

• update on timer resolution on windows from Rob Arnold.
  • Done, checked in earlier this week.

• Talos results
  • What's it look like for 1.8 -> 1.9?

• Todo: verification of test results
  • Fonts, valid rendering, etc.
  • TODO: Stuart: verify that concerns about talos results are addressed

• probe update
  • stan taking over mozilla/gecko side of probe work
    • will get Sun's dtrace patch as well as xp-probes checked in to new toplevel dir, then work on merging the two
  • crowder taking spidermonkey side
    • will be dtrace only
  • module owners should let stan know if they have any situations that they'd like to analyze, so that he has more information about how to structure the probe work.
    • for example, examining loading from network performance vs. loading from disk
  • Sun guys will be here on Aug 30 to do a presentation on dtrace, followed by hands-on work with a smaller group of engineers

Previous weeks

• jrpof tinderbox

• synthetic tests (TODO)
  • put specific tests into talos framework
  • poach mochitest performance tests
  • individual reftest-like items

• need to get a bunch of people looking at profiles (TODO)
  • take up some of these meetings to sit down and look at profiles
  • timer-based profiling is better (vtune/jprof/oprofile/etc., not quantify)
  • TODO: vlad to generate profile for next week's meeting

• running without Fx chrome
  • new pageloader stuff can do this

• examining default theme for performance issues

• Tp vs. Tp2
  • new pageloader stuff avoids this problem
  • Tp2 was flawed in some ways -- one big one is that it loaded pages in an iframe, so initial paint delay was never coming into play, and we were potentially reflowing more thannecessary

Other Information

Related Bugs