Ubiquity Study

• Champion: Jono (jdicarlo@mozilla.com)
• Status: Draft
• Duration: 1 week
• Implementation: Not started yet. Will be implemented as a feature inside Ubiquity extension.
• Data to collect: Frequency of command choices correlated with cleaned-up input strings. Should be no PII required.

Introduction

Ubiquity is Labs experiment in natural language interfaces. It is a time-saving extension that lets users run commands to simplify common web activities. Ubiquity suggests applicable commands based on the user's selection or natural language input. The key to making Ubiquity more humane and usable is likely in providing useful suggestions; therefore, this study will gather data on which Ubiquity commands are used most often, and what input users are typing into Ubiquity to access these commands, so that we can improve the quality of the suggestions.

Details

The first thing we'd like to know is the pure frequency of command usage across all users -- what is the most often used command, second most often used command, and so on. What does the distribution look like? This information will help us figure out where to put developer resources when improving standard commands. It will also let us know if any 3rd-party commands are being widely used, and if so which ones. There is no personal information in this data set since it is simply a list of commands and the number of times each was used.

The second thing we'd like to know is the mapping between the input that is entered and the command that is chosen. This is to help us evaluate the quality of our suggestions and figure out how to make better suggestions. Ideally, the command that people want will most often be at the top of the list. If some commands that are frequently used are often showing up as the fourth or fifth suggestion on the list and people are having to use the arrow keys to get down to them, then we know the suggestions must be improved in that area. For this part of the experiment we will need to collect input strings. Input strings may include arguments which contain personal information (such as addresses, email addresses, URLs, etc.) Therefore we must ensure that this information is all stripped out before any of it is sent to Mozilla. We will do this by removing all argument strings from the input and replacing them with the name of the argument data type. All personal information is in argument values, and don't need to know any of the argument values for our experiment. So for instance, if your input was "email hello to jinghua@mozilla.com", this would be recorded as "email (text) to (email address)". That way we get what we need for the experiment without ever seeing personal information.