Mobile/Gonzales: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 190: Line 190:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Original sample size (kB) !! New sample size (kB) !! Average compression (%) !! Total time (s)
! Original sample size (kB) !! New sample size (kB) !! Average compression (%) !! Total time (s) !! Median time (s)
|-
|-
| 25162 || 1177 || 95 || 25
| 25162 || 1177 || 95 || 25 || 2.46
|-
|}
 
{| class="wikitable"
|-
! Image size !! Time (s)
|-
| 531K ||  0.301
|-
| 844K ||  0.719
|-
| 992K ||  0.847
|-
| 1.1M ||  2.726
|-
| 1.3M ||  2.199
|-
| 1.5M ||  2.831
|-
| 1.7M ||  1.497
|-
| 3.5M ||  5.223
|-
| 4.5M ||  2.777
|-
| 8.0M ||  6.604
|-
|-
|}
|}

Revision as of 17:16, 30 May 2014

Speedy-gonzales.jpg

Gonzales is a project to find and evaluate ways to enable more efficient mobile browsing.

Goals

The goal of Gonzales is to verify whether we can enhance the mobile browsing experience by achieving the following improvements in a practical way:

  • Reduce radio time
  • Reduce page load times
  • Reduce bandwidth requirements
  • Increase user privacy
  • Increase responsiveness for slow sites

Ways we hope to accomplish some of these things:

  • Recompress images to be smaller without noticeable visual impact
  • Compress text (js, html) streams (gzip, bzip2, lzma?)
  • Caching
  • Reduce HTTPS round trips
  • Move safe browsing support to the proxy
  • Prefetch links and DNS resolutions

Other crazy stuff we could do:

  • Man-in-the-middle HTTPS traffic only for images (so they get recompressed) with user consent
  • Convert animated GIF to H264/WebM/whatever. gfycat.com does this now with apparently good results
  • Pre-Shumwayize Flash content
  • Automatic readability mode for some sites
  • Ad blocking
    • Only obnoxious ads?
    • Replace with our own?
    • Only block ones that track?
  • Add support for adaptive streaming (MPEG-DASH) for servers that don't support it. Transcode to lower bitrates.
    • Proxy HLS sites as MPEG-DASH
  • Opera Mini-like pre-rendering of pages

Background

HTTP

TODO

Transport Layer Security (TLS)

TLS handshake

Time [ms]             Client       Server
30                       SYN  -->                           
60                            <--  SYN ACK
90                       ACK  -->
120                           <--  Certificate
150         ChangeChiperSpec  --> 
180                           <--  ChangeCipherSpec
210                     Data  -->   
240                           <--  Data

SPDY

TODO

WiFi

TODO

3G/4G

LTE Mobile Network

Latency [ms]   Type            Device      RAN      Core Network             Internet
                                                    SGW     MME     PGW           
<100           control plane          <--> 
<5             user plane             <-->
30-100         backbone                        <--> 
variable       routing                                                  <-->

Control Plane Latency

               GPRS    EDGE    HSPA   HSPA+   LTE    LTE-Advanced
Latency [ms]   <1000   <1000   <500   <200    <100   <50

Radio Access Network (RAN)

TODO

Radio Resource Controller (RRC)

TODO

Packet Gateway (PGW)

TODO

Serving Gateway (SGW)

TODO

Mobility Management Entity (MME)

TODO

Image Compression

Image Formats

Format Lossy Transparency support Supported by Firefox
JPEG Yes No Yes
JPEG XR Yes Yes No
JPEG 2000 Yes Yes No
PNG No Yes Yes
JNG Yes Yes No
WebP Yes Yes No
BMP No Yes Yes

Sample

The sample used for the tests below was made using images from 20 different webpages from the Alexa's top 200 most visited websites list.

Format Number of images Total size (MB)
JPEG 325 12
PNG 817 14
GIF 325 1.8

PNG Compression

The PNG sample size is 12383KB.

Compression Method New sample size (kB) Average compression (%) Total time (s)
optipng -o 1 11560 7 70
optipng -o 5 10940 12 481
pngquant 5169 59 69
pngcrush 10997 12 90
pngcrush + optipng -o 5 10280 17 541
WebP (quality = 90) 4741 62 47
WebP (quality = 100) 8321 33 48

JPEG Compression

The sample size is 10846kB.

Compression Method New sample size (kB) Average compression (%) Total time (s)
mozjpeg 9663 11 53
mozjpeg (quality = 90) 8571 21 -
jpegoptim 10497 4 39
WebP (quality = 100) 17551 -61 42
WebP (quality = 90) 8316 24 40

GIF Compression

The sample size is 846kB.

Compression Method New sample size (kB) Average compression (%) Total time (s)
gifsicle 805 5 16

Animated GIF Compression

It is possible to convert animated GIFs to WebM videos (as gfycat.com does), this can be done with a straightforward ffmpeg command. The processing time can be high but the size reduction is huge!

Some stats from a small set of 10 animated GIFs:

Original sample size (kB) New sample size (kB) Average compression (%) Total time (s) Median time (s)
25162 1177 95 25 2.46
Image size Time (s)
531K 0.301
844K 0.719
992K 0.847
1.1M 2.726
1.3M 2.199
1.5M 2.831
1.7M 1.497
3.5M 5.223
4.5M 2.777
8.0M 6.604

Common Misconceptions

Losing radio link means losing TCP connection

The PGW terminates the TCP/UDP connections, application level connectivity is not tied to the physical radio link.

A device session is tied to an IP

The PGW provides NAT services, a device can be mapped to multiple IP/port combinations or multiple devices can share the same IP.

Bandwidth is the decisive factor for page load speed

Bandwidth gives the upper bound on transmission speed under optimal conditions. In realistic environments, considering the speed of light and the shortcomings of the transmission protocol, latency is the limiting factor for the maximum throughput.

Tracking Bugs

The Node.js based proxy prototype is maintained on GitHub: node-gonzales. Please use it to report issues and make pull requests.

Required features that Gonzales depends on are tracked in

  • bug 378637 - Add support for connecting to HTTP proxy over HTTPS
  • bug 366559 - Firefox/Gecko should support LZMA as an HTTP transfer-encoding method
  • bug 1010068 - Disable OCSP on Firefox for Android

Known issues affecting Gonzales are tracked in

  • bug 1014589 - Fennec crashes on page load when connected with SPDY proxy

System Overview

Gonzales is a secure proxy server that accepts requests via the SPDY protocol and serves compressed content directly from its cache, if available. 

Browser <-SPDY-> Gonzales                                <-HTTP/HTTPS/SPDY-> Web
               |                                         |
               Proxy <-> Cache <-?-> Compressor <-> Loader

Routing requests through a SPDY proxy does require less open TCP connections (multiplexing), decreases packet sizes (header compression) and reduces the number of packets (header caching) compared to HTTP/1. Low bandwidth and high latency connections should benefit from this.

The current trend on the Web shows an increase in average page size, especially due to high-resolution images. Additional image compression and downsizing should decrease the bandwidth requirements and enable faster page loads. To counteract the introduced processing overhead and boost load times for slow pages, we add intermediate caching for the compressed results.

Experimental Setups

To evaluate the system, we test each component in isolation first. We identify third-party systems suitable for integration into the prototype and evaluate them. All qualified systems will be considered for integration or as the base of our final implementation.

SPDY Proxy

Currently, only Google Chrome supports secure SPDY proxies, but it requires valid certificates, which makes it unsuitable for testing. We bypass the missing support by using shrpx in client mode as a forward proxy, and that way simulate SPDY support on any client.

Dependencies:

Here are some experimental setups.

P1

In this setup, we connect a desktop browser via HTTP with a local forwarding proxy, which establishes a SPDY connection with the remote secure SPDY proxy. The secure SDPY proxy connects to a local web proxy to fetch the documents. 

Desktop Client               <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
                             |      |            |      |           |
Firefox <-HTTP-> [shrpx -k -p]      --[shrpx -s]--      ----Squid----

P2

This setup is similar to P1, but running on a mobile client. This requires either direct secure SPDY proxy support on the mobile browser, or the usage of a local forwarding proxy running on the mobile device. 

Mobile Client               <-SPDY-> SPDY Proxy <-HTTP-> Web Proxy <-HTTP-> Web
                            |      |            |      |           |
Fennec <-HTTP-> [shrpx -k -p]      --[shrpx -s]--      ----Squid----

Compression

TODO

Caching

TODO

Performance Analysis

Let's collect some ideas here how to profile our prototypes.

Could be useful:

References

Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Gonzales&oldid=984409"