|
|
| (3 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| [[File:Speedy-gonzales.jpg|frameless]]
| |
|
| |
|
| Gonzales is a project to find and evaluate ways to enable more efficient mobile browsing.
| |
|
| |
| == Goals ==
| |
| The goal of Gonzales is to verify whether we can enhance the mobile browsing experience by achieving the following improvements in a practical way:
| |
| * Reduce radio time
| |
| * Reduce page load times
| |
| * Reduce bandwidth requirements
| |
| * '''Increase user privacy'''
| |
| * Increase responsiveness for slow sites
| |
|
| |
| Ways we hope to accomplish some of these things:
| |
| * Recompress images to be smaller without noticeable visual impact
| |
| * Compress text (js, html) streams (gzip, bzip2, lzma?)
| |
| * Caching
| |
| * Reduce HTTPS round trips
| |
| * Move safe browsing support to the proxy
| |
| * Prefetch links and DNS resolutions
| |
|
| |
| Other crazy stuff we could do:
| |
| * Man-in-the-middle HTTPS traffic only for images (so they get recompressed) with user consent
| |
| * Convert animated GIF to H264/WebM/whatever. gfycat.com does this now with apparently good results
| |
| * Pre-Shumwayize Flash content
| |
| * Automatic readability mode for some sites
| |
| * Ad blocking
| |
| ** Only obnoxious ads?
| |
| ** Replace with our own?
| |
| ** Only block ones that track?
| |
| * Add support for adaptive streaming (MPEG-DASH) for servers that don't support it. Transcode to lower bitrates.
| |
| ** Proxy HLS sites as MPEG-DASH
| |
| * Opera Mini-like pre-rendering of pages
| |
|
| |
| == Background ==
| |
| === HTTP ===
| |
| ''TODO''
| |
|
| |
| === Transport Layer Security (TLS) ===
| |
| ==== TLS handshake ====
| |
| <code>
| |
| '''Time''' [ms] '''Client''' '''Server'''
| |
| 30 SYN -->
| |
| 60 <-- SYN ACK
| |
| 90 ACK -->
| |
| 120 <-- Certificate
| |
| 150 ChangeChiperSpec -->
| |
| 180 <-- ChangeCipherSpec
| |
| 210 Data -->
| |
| 240 <-- Data
| |
| </code>
| |
|
| |
| === SPDY ===
| |
| ''TODO''
| |
|
| |
| === WiFi ===
| |
| ''TODO''
| |
|
| |
| === 3G/4G ===
| |
| ==== LTE Mobile Network ====
| |
| <code>
| |
| '''Latency''' [ms] '''Type''' '''Device''' '''RAN''' '''Core Network''' '''Internet'''
| |
| '''SGW''' '''MME''' '''PGW'''
| |
| <100 control plane <-->
| |
| <5 user plane <-->
| |
| 30-100 backbone <-->
| |
| variable routing <-->
| |
| </code>
| |
|
| |
| ==== Control Plane Latency ====
| |
| <code>
| |
| '''GPRS EDGE HSPA HSPA+ LTE LTE-Advanced'''
| |
| '''Latency''' [ms] <1000 <1000 <500 <200 <100 <50
| |
| </code>
| |
|
| |
| ==== Radio Access Network (RAN) ====
| |
| ''TODO''
| |
|
| |
| ==== Radio Resource Controller (RRC) ====
| |
| ''TODO''
| |
|
| |
| ==== Packet Gateway (PGW) ====
| |
| ''TODO''
| |
|
| |
| ==== Serving Gateway (SGW) ====
| |
| ''TODO''
| |
|
| |
| ==== Mobility Management Entity (MME) ====
| |
| ''TODO''
| |
|
| |
| === Image Compression ===
| |
|
| |
| ==== Image Formats ====
| |
|
| |
| {| class="wikitable"
| |
| |-
| |
| ! Format !! Lossy !! Transparency support !! Supported by Firefox
| |
| |-
| |
| | JPEG || Yes || No || Yes
| |
| |-
| |
| | JPEG XR || Yes || Yes || No
| |
| |-
| |
| | JPEG 2000 || Yes || Yes || No
| |
| |-
| |
| | PNG || No || Yes || Yes
| |
| |-
| |
| | JNG || Yes || Yes || No
| |
| |-
| |
| | WebP || Yes || Yes || No
| |
| |-
| |
| | BMP || No || Yes || Yes
| |
| |-
| |
| |}
| |
|
| |
| ==== Sample ====
| |
|
| |
| The sample used for the tests below was made using images from 20 different webpages from the Alexa's top 200 most visited websites list.
| |
|
| |
| {| class="wikitable"
| |
| |-
| |
| ! Format !! Number of images !! Total size (MB)
| |
| |-
| |
| | JPEG || 325 || 12
| |
| |-
| |
| | PNG || 817 || 14
| |
| |-
| |
| | GIF || 325 || 1.8
| |
| |}
| |
|
| |
| ==== PNG Compression ====
| |
|
| |
| The PNG sample size is 12383KB.
| |
|
| |
| {| class="wikitable sortable"
| |
| |-
| |
| ! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
| |
| |-
| |
| | optipng -o 1 || 11560 || 7 || 70
| |
| |-
| |
| | optipng -o 5 || 10940 || 12 || 481
| |
| |-
| |
| | [http://pngquant.org/ pngquant] || 5169 || 59 || 69
| |
| |-
| |
| | pngcrush || 10997 || 12 || 90
| |
| |-
| |
| | pngcrush + optipng -o 5 || 10280 || 17 || 541
| |
| |-
| |
| | WebP (quality = 90) || 4741 || 62 || 47
| |
| |-
| |
| | WebP (quality = 100) || 8321 || 33 || 48
| |
| |-
| |
| |}
| |
|
| |
| ==== JPEG Compression ====
| |
|
| |
| The sample size is 10846kB.
| |
|
| |
| {| class="wikitable sortable"
| |
| |-
| |
| ! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
| |
| |-
| |
| | mozjpeg || 9663 || 11 || 53
| |
| |-
| |
| | mozjpeg (quality = 90) || 8571 || 21 || -
| |
| |-
| |
| | jpegoptim || 10497 || 4 || 39
| |
| |-
| |
| | WebP (quality = 100) || 17551 || -61 || 42
| |
| |-
| |
| | WebP (quality = 90) || 8316 || 24 || 40
| |
| |-
| |
| |}
| |
|
| |
| ==== GIF Compression ====
| |
|
| |
| The sample size is 846kB.
| |
|
| |
| {| class="wikitable sortable"
| |
| |-
| |
| ! Compression Method !! New sample size (kB) !! Average compression (%) !! Total time (s)
| |
| |-
| |
| | gifsicle || 805 || 5 || 16
| |
| |-
| |
| |}
| |
|
| |
| ==== Animated GIF Compression ====
| |
|
| |
| It is possible to convert animated GIFs to WebM videos (as [http://gfycat.com/ gfycat.com] does), this can be done with a straightforward ffmpeg command. The processing time can be high but the size reduction is huge!
| |
|
| |
| Some stats from a small set of 10 animated GIFs:
| |
| {| class="wikitable"
| |
| |-
| |
| ! Original sample size (kB) !! New sample size (kB) !! Average compression (%) !! Total time (s) !! Median time (s)
| |
| |-
| |
| | 25162 || 1177 || 95 || 25 || 2.46
| |
| |-
| |
| |}
| |
|
| |
| {| class="wikitable"
| |
| |-
| |
| ! Image size !! Time (s)
| |
| |-
| |
| | 531K || 0.301
| |
| |-
| |
| | 844K || 0.719
| |
| |-
| |
| | 992K || 0.847
| |
| |-
| |
| | 1.1M || 2.726
| |
| |-
| |
| | 1.3M || 2.199
| |
| |-
| |
| | 1.5M || 2.831
| |
| |-
| |
| | 1.7M || 1.497
| |
| |-
| |
| | 3.5M || 5.223
| |
| |-
| |
| | 4.5M || 2.777
| |
| |-
| |
| | 8.0M || 6.604
| |
| |-
| |
| |}
| |
|
| |
| === Common Misconceptions ===
| |
| ==== Losing radio link means losing TCP connection ====
| |
| The PGW terminates the TCP/UDP connections, application level connectivity is not tied to the physical radio link.
| |
|
| |
| ==== A device session is tied to an IP ====
| |
| The PGW provides NAT services, a device can be mapped to multiple IP/port combinations or multiple devices can share the same IP.
| |
|
| |
| ==== Bandwidth is the decisive factor for page load speed ====
| |
| Bandwidth gives the upper bound on transmission speed under optimal conditions. In realistic environments, considering the speed of light and the shortcomings of the transmission protocol, ''latency'' is the limiting factor for the maximum throughput.
| |
|
| |
| == Tracking Bugs ==
| |
| The Node.js based proxy prototype is maintained on GitHub: [https://github.com/eamsen/node-gonzales node-gonzales]. Please use it to report issues and make pull requests.
| |
|
| |
| Required '''features''' that Gonzales depends on are tracked in
| |
| * {{bug|378637}} - Add support for connecting to HTTP proxy over HTTPS
| |
| * {{bug|366559}} - Firefox/Gecko should support LZMA as an HTTP transfer-encoding method
| |
| * {{bug|1010068}} - Disable OCSP on Firefox for Android
| |
|
| |
| Known '''issues''' affecting Gonzales are tracked in
| |
| * {{bug|1014589}} - Fennec crashes on page load when connected with SPDY proxy
| |
|
| |
| == System Overview ==
| |
| Gonzales is a secure proxy server that accepts requests via the SPDY protocol and serves compressed content directly from its cache, if available.
| |
|
| |
| <code>
| |
| Browser <-SPDY-> Gonzales <-HTTP/HTTPS/SPDY-> Web
| |
| | |
| |
| Proxy <-> Cache <-?-> Compressor <-> Loader
| |
| </code>
| |
|
| |
| Routing requests through a SPDY proxy does require less open TCP connections (multiplexing), decreases packet sizes (header compression) and reduces the number of packets (header caching) compared to HTTP/1. Low bandwidth and high latency connections should benefit from this.
| |
|
| |
| The current trend on the Web shows an increase in average page size, especially due to high-resolution images. Additional image compression and downsizing should decrease the bandwidth requirements and enable faster page loads. To counteract the introduced processing overhead and boost load times for slow pages, we add intermediate caching for the compressed results.
| |
|
| |
| == Experimental Setups ==
| |
| To evaluate the system, we test each component in isolation first.
| |
| We identify third-party systems suitable for integration into the prototype and evaluate them. All qualified systems will be considered for integration or as the base of our final implementation.
| |
|
| |
| === SPDY Proxy ===
| |
| Currently, only Google Chrome supports secure SPDY proxies, but it requires valid certificates, which makes it unsuitable for testing. We bypass the missing support by using '''shrpx''' in client mode as a ''forward proxy'', and that way simulate SPDY support on any client.
| |
|
| |
| Dependencies:
| |
| * Firefox/Fennec
| |
| * Spdylay (http://tatsuhiro-t.github.io/spdylay)
| |
| * Squid (http://www.squid-cache.org)
| |
| * [alternative] Node SPDY proxy (https://github.com/igrigorik/node-spdyproxy)
| |
|
| |
| Here are some experimental setups.
| |
|
| |
| ==== P1 ====
| |
| In this setup, we connect a desktop browser via HTTP with a local forwarding proxy, which establishes a SPDY connection with the remote secure SPDY proxy. The secure SDPY proxy connects to a local web proxy to fetch the documents.
| |
|
| |
| <code>
| |
| Desktop Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| |
| | | | | |
| |
| Firefox <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
| |
| </code>
| |
|
| |
| ==== P2 ====
| |
| This setup is similar to P1, but running on a mobile client. This requires either direct secure SPDY proxy support on the mobile browser, or the usage of a local forwarding proxy running on the mobile device.
| |
|
| |
| <code>
| |
| Mobile Client <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
| |
| | | | | |
| |
| Fennec <-HTTP-> [shrpx -k -p] --[shrpx -s]-- ----Squid----
| |
| </code>
| |
|
| |
| === Compression ===
| |
| ''TODO''
| |
|
| |
| === Caching ===
| |
| ''TODO''
| |
|
| |
| == Performance Analysis ==
| |
| Let's collect some ideas here how to profile our prototypes.
| |
|
| |
| Could be useful:
| |
| * [https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests Talos page load tests]
| |
|
| |
| == References ==
| |
| * [http://www.ietf.org/rfc/rfc2186.txt Internet Cache Protocol (ICP)]
| |
| * [http://www.ietf.org/rfc/rfc2187.txt Application of Internet Cache Protocol (ICP)]
| |
| * [http://tools.ietf.org/html/rfc2616 HTTP/1.1]
| |
| * [http://tools.ietf.org/id/draft-agl-tls-nextprotoneg-00.txt Transport Layer Security (TLS) Next Protocol Negotiation Extension (NPN)]
| |
| * [http://www.chromium.org/spdy/spdy-proxy Chromium SPDY Proxy]
| |
| * [http://tatsuhiro-t.github.io/spdylay/ Spdylay - SPDY C Library]
| |
| * [https://github.com/igrigorik/node-spdyproxy Node.js SPDY Forwarding Proxy]
| |
| * [http://nginx.org/ NGINX - Reverse Proxy]
| |
| * [http://developer.att.com/application-resource-optimizer AT&T ARO]
| |
| * [http://www.igvita.com/posa/high-performance-networking-in-google-chrome/ High Performance Networking in Google Chrome]
| |
| * [http://commondatastorage.googleapis.com/io-2013/presentations/263%20-%20io-radio-performance.pdf Mobile Performance from Radio Up] ([https://developers.google.com/events/io/sessions/327864330 video])
| |
| * [http://www.igvita.com/2012/06/25/spdy-and-secure-proxy-support-in-google-chrome/ Secure SPDY Proxy Support in Google Chrome]
| |
| * [http://amazonsilk.wordpress.com/2011/09/28/introducing-amazon-silk/ Amazon Silk Browser]
| |
| * [http://chimera.labs.oreilly.com/books/1230000000545 High Performance Browser Networking]
| |
| * [https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-ssl/ Experience report on CDNs and SPDY]
| |
| * [https://developers.google.com/speed/docs/best-practices/caching Caching best practices]
| |