Gaia/Email/SecretDebugMode

It's a secret to everybody... who has not read this wiki page.

What's so cool about the secret debug menu?

The secret debug menu exposes logging functionality for debugging and diagnosis. It produces JSON log files.

By default, logging starts out in a minimal no-op mode. Events/errors are counted, everything else is discarded. There are two additional modes of logging: "general", and test mode. "general" is what gets enabled in the e-mail app and is intended to only log non-sensitive data. In test mode, additional arguments are logged to provide a more comprehensive understanding of the system is doing.

For example, the log definition for the IMAP protocol connection, eliding other entry definitions etc, looks like:

 var LOGFAB = exports.LOGFAB = $log.register(module, {
   ImapProtoConn: {
     events: {
       data: { length: false },
     },
     TEST_ONLY_events: {
       // This may be a Buffer and therefore need to be coerced
       data: { data: $log.TOSTRING },
     },
 }); // end LOGFAB

The logging code then looks like:

 if (self._LOG) self._LOG.data(chunk.length, chunk);

In the default and "general" logging modes, 'chunk' gets ignored and is not touched by the logging method. Only when run in unit test mode is the data captured.

Toggling the setting changes the configuration in the IndexedDB database which is loaded and takes effect only at app startup. When enabled, a setInterval callback is added that fires every 1 second to check for new log entries and put them in a bin allocated to that second. If there were any entries, they are added to a list that is used to only keep the 30 most recent seconds with log entries.

Getting to the Secret Debug Menu

• Bring up the settings page.
  • The settings page is only accessible once an account has been added, so you may need to add a fake account. In the event we have removed the fake account button, you can manually create a fake account by creating an e-mail account like "foo@example.com". It will not require you to authenticate, but is not very realistic.
  • The settings page is accessed via the gear icon on the folder list page. You access the folder list by clicking the parallel lines icon in the upper left.
• On the settings page, click the whitespace vertically below the "Add Account" button 5 times within 2 seconds. (The sensitized whitespace is about the size of the "add account" button, but starts just beneath the last horizontal line on the screen.)

Enabling Logging that does not entrain user data (general mode)

• (Bring up the secret debug menu. See above.)
• Turn on logging:
  • v2.0 and later: Tap the combo-box/select element with the label "Logging Mode" above it. Prior to tapping it, the select element will indicate the current logging level which is probably "No logging." Once you tap it, it will bring up the list of logging levels. You want "Circular: Safe data: no passwords or message contents logged." which should be the second item in the list. Tap it and it should turn blue and get a checkmark. Then tap the "OK" button at the bottom of the screen.
  • pre-v2.0: Turn on logging by pressing the button that says "Logging is DISABLED; toggle". It will change to say "Logging is ENABLED; toggle" if the click was processed.
• Reset the application by pushing the "Reset app" button. This is required in order to force the application to restart with logging active.

An example of general logging can be seen here: pretty UI raw JSON prettified JSON

Enabling logging that DOES ENTRAIN USER DATA (test mode)

This captures a lot of data because it is intended to really only be enabled for unit tests. While some layers of the system like the IMAP protocol are careful to avoid logging any passwords, not all layers do. For example, the communications between the front-end and back-end is currently entirely logged, so if you create a new account while this level of logging is active, YOUR PASSWORD WILL BE LOGGED. Even if you don't create a new account, IMAP protocol traffic will be logged, so the contents of messages will be logged!

• (Bring up the secret debug menu using the instructions two sections before this one)
• Turn on logging:
  • v2.0 and later: Tap the combo-box/select element with the label "Logging Mode" above it. Prior to tapping it, the select element will indicate the current logging level which is probably "No logging." Once you tap it, it will bring up the list of logging levels. You want "Circular: Dangerous data: passwords and message contents logged." which should be the third item in the list. Tap it and it should turn blue and get a checkmark. Then tap the "OK" button at the bottom of the screen.
  • pre-v2.0: Enable logging as described in the "general" mode case. Then, turn on test mode by clicking on the button that says "logging is SAFE; toggle". It should now read "logging DANGEROUSLY ENTRAINS USER DATA; toggle".
• Reset the application by pushing the "Reset app" button. This is required in order to force the application to restart with the new logging settings active.

An example of a unit test run which operates at test levels of detail can be seen here. The actual test data will be structured time-wise more like the general examples.

Dumping log files to disk

We write log files to device storage so you can grab them via adb using the sdcard permission. We name our files along the lines of gem-log-TIMESTAMP.json where "TIMESTAMP" is Date.now().

• (Bring up the secret debug menu)
• Press the button labeled "Dump log to storage".
• Your file should now be named gem-log-TIMESTAMP.json and be in devicestorage somewhere. There should be a console.log that tells you the path to the file that was logged.

Getting the log files out

Here is a script that works for me on a computer with a Flame v2.0 device attached to grab the file to /tmp/gem-log.json:

 #!/bin/bash
 set -o errexit
 set -o pipefail
 
 logpath=`adb shell ls /storage/sdcard0/gem-log-*.json | head -1 | tr -d '\n\r'`
 echo I am grabbing: $logpath
 adb pull ${logpath} /tmp/gem-log.json
 adb shell rm $logpath
 chmod a+r /tmp/gem-log.json

Here is the same script that works for me on a Flame v2.2 device (the sdcard mount point changed):

 #!/bin/bash
 set -o errexit
 set -o pipefail
 
 logpath=`adb shell ls /sdcard/gem-log-*.json | head -1 | tr -d '\n\r'`
 echo I am grabbing: $logpath
 adb pull ${logpath} /tmp/gem-log.json
 adb shell rm $logpath
 chmod a+r /tmp/gem-log.json

Here is a script that works for me when using b2g-desktop on linux:

 #!/bin/bash
 set -o errexit
 set -o pipefail
 
 # maybe grep instead of
 logpath=`ls ~/Videos/gem-log-*.json | head -1 | tr -d '\n\r'`
 echo I am grabbing: $logpath
 mv ${logpath} /tmp/gem-log.json
 chmod a+r /tmp/gem-log.json

Interpreting / viewing the log files

You may just want to provide the JSON file to the developers of the e-mail application. But if you don't, here is what you do:

Options:

• Use the pre-built one.
  • Have the file accessible via an http server on localhost and make sure your CORS setup is friendly:
  • Paste http://clicky.visophyte.org/tools/arbpl-standalone/?log= into your URL bar, followed by your localhost URL.
  • (Alternately, you can just download the contents of http://clicky.visophyte.org/tools/arbpl-standalone/ to disk and then access the file locally via "file://" URL.
• Build your own: Check out arbitrarypushlog
  • use your host's own localhost server to expose the UI. Specifically, symlink or what not your git checkout of ArbPL.
    • use the development mode, no building required. Use index-app.html with "?log=/PATH/TO/LOG/ON/THE/WEBSERVER".
    • build the optimized standalone mode,then use index-standalone.html with a "?log=PATHTOLOG" argument, keeping in mind that you may need to host both the source and the log file at the same origin in order for things to work if the log file is not served with blanket CORS permissions.