Mozilla.com/2011-Rebranding-Project/Benchmarks

Benchmarks

Merging the mozilla.org codebase into mozilla.com is very tricky. We are simply hacking the two codebases to run under one domain with Apache and PHP magic. There are several ways to do this, but we must preserve integrity and performance of all pages.

In order to justify our Apache+PHP solution, here is some raw data of why this merge should be successful.

How to read these graphs

These graphics were generated using the Apache benchmark tool `ab`. They show the percentage of requests that were returned in a certain response time in milliseconds. This is a decent indication of the performance of a site.

These were run with a concurrency rate of 10 and for 5000 requests which is generous enough to get a realistic picture. That means that it ran 10 connections at the same time (opening new ones when old ones finished) until 5000 requests were finished.

It's *very* difficult to accurately benchmark a site, but I ran these several times and found my setup to produce somewhat accurate, predictable results. Don't read too much into these, they should just give a general picture.

Key

Terms used in the keys on these graphs:

• RewriteMap+PHP - RewriteMap redirects and PHP magic to serve mozilla.org as top-level URLs
• Apache Redirect+PHP - Raw .htaccess redirects (copied from mozilla.org's htaccess) and PHP magic for mozilla.org pages
• PHP - PHP magic for both htaccess redirects and mozilla.org pages
• Pure - Normal mozilla.com or mozilla.org site as it stands now

htaccess redirects

mozilla.org had about 1500 rewrites/redirects in its .htaccess file. We need to port these over somehow, but with that many redirects performance is a major concern. The concern is that every request must process the entire htaccess file, meaning every request would incur a decent performance hit if we don't do this right.

There 4 different ways to port the redirects:

• RewriteMap(txt)+PHP - Use Apache's RewriteMap directive which loads optimized redirects from a text file
• RewriteMap(dbm)+PHP - Use Apache's RewriteMap directive which loads optimized redirects from a database file
• Apache Redirect+PHP - Simply move over the 1500 Redirect/Rewrite lines, use PHP to load in org pages
• PHP - Implement the redirects in PHP, where the real .org pages are handled

The "Pure" line tests hitting the mozilla.org codebase as it stands today, for reference.

Here are the results tested with a concurrency of 10 for 15000 requests:

If we zoom in some by only testing 5000 requests and increase concurrency to 30:

Conclusion: Obviously RewriteMap is the way to go. It's worth noting that "Apache Redirect+PHP" was the only one that must be avoided since every request would suffer due to the ballooning of htaccess. The other methods can selectively be applied only to pages that don't exist.

We've actually improved the mozilla.org pages' performance a lot since we removed all the htaccess redirects, as we will see in another graph.

Lastly, we can compile the RewriteMap text file into a database if needed, as it performs better under high load. However, the insignificant gains aren't worth worrying about that right now.

mozilla.com PHP page

Let's benchmark a normal mozilla.com PHP page: /en-US/firefox/new/.