https://wiki.mozilla.org/api.php?action=feedcontributions&user=Wlach&feedformat=atomMozillaWiki - User contributions [en]2024-03-29T09:45:25ZUser contributionsMediaWiki 1.27.4https://wiki.mozilla.org/index.php?title=Data_in_the_Open&diff=1238363Data in the Open2021-10-12T12:54:28Z<p>Wlach: Data in the open</p>
<hr />
<div>== Rationale ==<br />
<br />
The value of working in the open is at the core of Mozilla. This is clearly outlined in section 7 (“Free and open source software promotes the development of the internet as a public resource”) and section 8 (“ Transparent community-based processes promote participation, accountability and trust.”) of the [https://www.mozilla.org/en-CA/about/manifesto/ Mozilla Manifesto]. In the Data organization, we see the positive effects of these principles in our work:<br />
<br />
* Open practices (like the Data Stewardship model) builds trust with our users: this provides us social license to explore new areas like personalized search results and other contextual services.<br />
* Our involvement in the Outreachy and Google Summer of Code programs has helped us build out a first class data collection and analysis platform, improving the user experience of about:telemetry, metadata tools like the Glean Dictionary, and others.<br />
* Having the source to our systems publicly available can, in many cases, make working with partners easier since it obviates the need to sign NDAs and give privileged access to our systems.<br />
<br />
That said, Data has some unique characteristics which make living up to these principles challenging: there are privacy, legal, and business reasons why we can’t just release all of the data Mozilla collects (or processes) to the public in unaggregated form. While Mozilla goes out of its way not to collect personally identifiable information about our users when not required, in practice some of the data sets could be correlated to a particular individual given enough external resources and effort. Releasing this information carelessly is both a moral and legal risk. And in terms of the business, releasing certain types of data (for example, client-level search count information) would compromise Mozilla’s ability to compete in a cut-throat marketplace. That said, this is no reason to give up entirely. There is a balance between our core principles and these realities, and it tilts strongly towards openness.<br />
<br />
This document attempts to codify Mozilla Data’s existing practices around working in the open. It is intended to be a resource we can draw on when making a decision on a technology choice, working group setup, or any other decision about what we work on and how we work on it.<br />
<br />
== Best Practices ==<br />
<br />
=== Software and Tools ===<br />
<br />
We use open tools for software development wherever possible and reasonable. Examples of such tools include:<br />
<br />
* Public GitHub repositories in the Mozilla organization<br />
* Issue tracking using either GitHub issues (see above) or bugzilla.mozilla.org<br />
* Open source software, such as Python and JSONSchema<br />
<br />
It is acceptable to use private repositories or issue tracking tools (for example: JIRA) given a compelling business need or strong legal case, but this should be the exception with appropriate justification provided (e.g. inside a proposal for a new project, or inside the GitHub repository itself). <br />
<br />
Private repositories and Mozilla-internal-only tools make collaboration with external contributors (for example, Outreachy interns) essentially impossible: take extra care to avoid using them if this is something you want to allow, either now or in the future.<br />
<br />
=== Communication ===<br />
<br />
Likewise, we favour tools for communicating about our work where possible and convenient. Examples of this include:<br />
<br />
* Public working groups documented on [[Data|wiki.mozilla.org/Data]]<br />
* [[Matrix]] for synchronous chat (public by default: end-to-end encrypted rooms may be used to discuss matters under NDA, which enables collaboration with a wider group of contributors)<br />
* Public mailing lists using Google Groups or Mozilla’s instance of discourse for asynchronous discussions -- these tools are more inclusive of people in different timezones.<br />
* World-readable Google documents for proposals (linked to inside the appropriate working group)<br />
<br />
As of this writing, most discussion on Data topics currently happens on Slack which is a closed system: Slack’s support for threaded messaging is frequently cited as a strong technical reason to use this platform (this may be fixed in the future: [https://github.com/vector-im/element-web/issues/2349 vector-im/element-web#2349]). For pragmatic reasons, this is ok: but bear in mind conversations conducted on this platform exclude community involvement and make it more difficult to provide context to others about our work. We see Matrix as the future of open communication at Mozilla: the existing rooms around Glean (#glean, #glean-dev, and #glean-dictionary) provide good examples of how this can work well in practice.<br />
<br />
=== Open Datasets ===<br />
<br />
In addition to publishing the source code to our data collection and analysis systems, Mozilla also releases a [https://docs.telemetry.mozilla.org/cookbooks/public_data.html small set of aggregated datasets] of interest to the public, along with a [https://data.firefox.com/ public data report] (which has proven useful for answering both internal and external questions about the number and characteristics of Firefox users). Publishing our datasets and the dashboards we use to visualize them is perhaps the strongest way we can signal to the public that we think about this area differently, as well as empowering the entire Mozilla community to make data-informed decisions. See our [[Data_Publishing|data publishing policy]] for information on how we do this without compromising user privacy.</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1238362Data2021-10-12T12:26:28Z<p>Wlach: /* Policy */ data in the open</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Our approach to data is most succinctly described by the [https://www.mozilla.org/privacy/principles/ Mozilla Privacy Principles]. If you want to know what Mozilla thinks about data, the Principles will tell you that.<br />
<br />
From those principles come [https://www.mozilla.org/privacy/ Mozilla's Privacy Notices]. They differ from product to product because the data each product deals with is different. If you want to know what kinds of data each Mozilla product collects and what we do with it, the Privacy Notices will tell you that.<br />
<br />
From the Principles and the Notices we derive operational processes to allow us to make decisions about what data we can collect, store, and publish and how we go about it. Here are a few of them:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) of the data we collect for the public benefit<br />
* [[Data_in_the_Open|Data in the Open]]: How Data@Mozilla works in the open<br />
<br />
If you want to know how we ensure the data we collect, store, and publish abide by the Privacy Notices and the Principles, these processes will tell you that.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [[Data/WorkingGroups/GleanPlatform | Glean Platform]]<br />
* [[Data/WorkingGroups/GleanDictionary | Glean Dictionary]]<br />
* [[Data/WorkingGroups/CrashReporting | Crash Reporting Working Group]]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you're a community member and are looking to participate in our work, see the public working groups above. We also maintain a list of mentored issues on [https://codetribute.mozilla.org/projects/data codetribute.mozilla.org].<br />
<br />
If you are a Mozilla employee, information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1238302Data/WorkingGroups/GleanDictionary2021-10-06T13:58:35Z<p>Wlach: /* Charter */</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Production URL || [https://dictionary.telemetry.mozilla.org dictionary.telemetry.mozilla.org]<br />
|-<br />
| Development URL || [https://glean-dictionary-dev.netlify.app glean-dictionary-dev.netlify.app]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || No longer occurring (Glean Dictionary is occasionally discussed in the [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform] meetings). For legacy minutes, see: [https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] <br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (done) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (done) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
# (in progress) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
<br />
We are currently discussing what comes after this work is complete. Possible directions include integrations with Mozilla's Looker deploy and better support for Firefox Desktop (as [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html it moves over to Glean]).<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Getting involved =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Communication =<br />
<br />
There are two primary communication channels for the group:<br />
<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack). Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1238301Data/WorkingGroups/GleanDictionary2021-10-06T13:58:08Z<p>Wlach: Minor updates</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Production URL || [https://dictionary.telemetry.mozilla.org dictionary.telemetry.mozilla.org]<br />
|-<br />
| Development URL || [https://glean-dictionary-dev.netlify.app glean-dictionary-dev.netlify.app]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || No longer occurring (Glean Dictionary is occasionally discussed in the [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform] meetings). For legacy minutes, see: [https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] <br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (done) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (in progress) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (in progress) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
We are currently discussing what comes after this work is complete. Possible directions include integrations with Mozilla's Looker deploy and better support for Firefox Desktop (as [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html it moves over to Glean]).<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Getting involved =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Communication =<br />
<br />
There are two primary communication channels for the group:<br />
<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack). Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1238062Data/WorkingGroups/GleanDictionary2021-09-21T18:57:53Z<p>Wlach: /* Communication */</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.telemetry.mozilla.org dictionary.telemetry.mozilla.org]<br />
|-<br />
| Development URL || [https://glean-dictionary-dev.netlify.app glean-dictionary-dev.netlify.app]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || No longer occurring (Glean Dictionary is occasionally discussed in the [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform] meetings). For legacy minutes, see: [https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] <br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (done) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (in progress) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (in progress) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
We are currently discussing what comes after this work is complete. Possible directions include integrations with Mozilla's Looker deploy and better support for Firefox Desktop (as [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html it moves over to Glean]).<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Getting involved =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Communication =<br />
<br />
There are two primary communication channels for the group:<br />
<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1238061Data/WorkingGroups/GleanDictionary2021-09-21T18:57:34Z<p>Wlach: /* Quick Reference */ Remove meetings</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.telemetry.mozilla.org dictionary.telemetry.mozilla.org]<br />
|-<br />
| Development URL || [https://glean-dictionary-dev.netlify.app glean-dictionary-dev.netlify.app]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || No longer occurring (Glean Dictionary is occasionally discussed in the [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform] meetings). For legacy minutes, see: [https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] <br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (done) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (in progress) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (in progress) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
We are currently discussing what comes after this work is complete. Possible directions include integrations with Mozilla's Looker deploy and better support for Firefox Desktop (as [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html it moves over to Glean]).<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Getting involved =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Communication =<br />
<br />
There are three primary communication channels for the group:<br />
<br />
* A fortnightly Zoom meeting open to all, for synchronous discussion and larger-scale strategic planning<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1235018Data/WorkingGroups/GleanDictionary2021-04-15T15:48:14Z<p>Wlach: Various updates</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.protosaur.dev dictionary.protosaur.dev]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || Fortnightly cadence on Wednesdays at [http://www.timebie.com/std/utc.php?q=17 5pm UTC].<br />
<br />
Zoom Room: [https://mozilla.zoom.us/j/91218334714 91218334714]<br />
<br />
[https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] open to the public<br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (done) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (in progress) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (in progress) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
We are currently discussing what comes after this work is complete. Possible directions include integrations with Mozilla's Looker deploy and better support for Firefox Desktop (as [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html it moves over to Glean]).<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Getting involved =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Communication =<br />
<br />
There are three primary communication channels for the group:<br />
<br />
* A fortnightly Zoom meeting open to all, for synchronous discussion and larger-scale strategic planning<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1234866Data2021-04-06T14:10:53Z<p>Wlach: /* Data @ Mozilla */ codetribute</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Our approach to data is most succinctly described by the [https://www.mozilla.org/privacy/principles/ Mozilla Privacy Principles]. If you want to know what Mozilla thinks about data, the Principles will tell you that.<br />
<br />
From those principles come [https://www.mozilla.org/privacy/ Mozilla's Privacy Notices]. They differ from product to product because the data each product deals with is different. If you want to know what kinds of data each Mozilla product collects and what we do with it, the Privacy Notices will tell you that.<br />
<br />
From the Principles and the Notices we derive operational processes to allow us to make decisions about what data we can collect, store, and publish. Here are a few of them:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) of the data we collect for the public benefit<br />
<br />
If you want to know how we ensure the data we collect, store, and publish abide by the Privacy Notices and the Principles, these processes will tell you that.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [[Data/WorkingGroups/GleanPlatform | Glean Platform]]<br />
* [[Data/WorkingGroups/GleanDictionary | Glean Dictionary]]<br />
* [[Data/WorkingGroups/CrashReporting | Crash Reporting Working Group]]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you're a community member and are looking to participate in our work, see the public working groups above. We also maintain a list of mentored issues on [https://codetribute.mozilla.org/projects/data codetribute.mozilla.org].<br />
<br />
If you are a Mozilla employee, information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1234725Data/WorkingGroups/GleanDictionary2021-03-29T14:10:39Z<p>Wlach: /* Charter */ Update status</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.protosaur.dev dictionary.protosaur.dev]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || Fortnightly cadence on Wednesdays at [http://www.timebie.com/std/utc.php?q=17 5pm UTC].<br />
<br />
Zoom Room: [https://mozilla.zoom.us/j/91218334714 91218334714]<br />
<br />
[https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] open to the public<br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (in progress) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (not started) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (in progress) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
After phase 4 of the project is completed, we will re-evaluate the future of this group.<br />
<br />
= Communication =<br />
<br />
There are three primary communication channels for the group:<br />
<br />
* A fortnightly Zoom meeting open to all, for synchronous discussion and larger-scale strategic planning<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Membership =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1234724Data/WorkingGroups/GleanDictionary2021-03-29T14:09:09Z<p>Wlach: Metric annotations</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.protosaur.dev dictionary.protosaur.dev]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || Fortnightly cadence on Wednesdays at [http://www.timebie.com/std/utc.php?q=17 5pm UTC].<br />
<br />
Zoom Room: [https://mozilla.zoom.us/j/91218334714 91218334714]<br />
<br />
[https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] open to the public<br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (in progress) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (not started) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (planning) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
After phase 4 of the project is completed, we will re-evaluate the future of this group.<br />
<br />
= Communication =<br />
<br />
There are three primary communication channels for the group:<br />
<br />
* A fortnightly Zoom meeting open to all, for synchronous discussion and larger-scale strategic planning<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Membership =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1mUHlWltHExFg9fD6dTDV98nIzrV7wRGgMw8783HDNfI/edit#heading=h.26qa4tdenv6x Glean Metric Annotation Repository] || 22 February 2021 || Implemented<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1234223Data2021-03-01T14:37:39Z<p>Wlach: </p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Our approach to data is most succinctly described by the [https://www.mozilla.org/privacy/principles/ Mozilla Privacy Principles]. If you want to know what Mozilla thinks about data, the Principles will tell you that.<br />
<br />
From those principles come [https://www.mozilla.org/privacy/ Mozilla's Privacy Notices]. They differ from product to product because the data each product deals with is different. If you want to know what kinds of data each Mozilla product collects and what we do with it, the Privacy Notices will tell you that.<br />
<br />
From the Principles and the Notices we derive operational processes to allow us to make decisions about what data we can collect, store, and publish. Here are a few of them:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) of the data we collect for the public benefit<br />
<br />
If you want to know how we ensure the data we collect, store, and publish abide by the Privacy Notices and the Principles, these processes will tell you that.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform]<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1234222Data2021-03-01T14:37:14Z<p>Wlach: Put glean platform before glean dictionary</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Our approach to data is most succinctly described by the [https://www.mozilla.org/privacy/principles/ Mozilla Privacy Principles]. If you want to know what Mozilla thinks about data, the Principles will tell you that.<br />
<br />
From those principles come [https://www.mozilla.org/privacy/ Mozilla's Privacy Notices]. They differ from product to product because the data each product deals with is different. If you want to know what kinds of data each Mozilla product collects and what we do with it, the Privacy Notices will tell you that.<br />
<br />
From the Principles and the Notices we derive operational processes to allow us to make decisions about what data we can collect, store, and publish. Here are a few of them:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) of the data we collect for the public benefit<br />
<br />
If you want to know how we ensure the data we collect, store, and publish abide by the Privacy Notices and the Principles, these processes will tell you that.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanPlatform Glean Platform]<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233795Data2021-02-18T16:28:55Z<p>Wlach: /* Policy */ slight wording change</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Here are some of our policies around this:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) of the data we collect for the public benefit<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233794Data2021-02-18T16:27:31Z<p>Wlach: /* Data @ Mozilla */ remove extra newline</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Here are some of our policies around this:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) our data for the public benefit<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233793Data2021-02-18T16:22:15Z<p>Wlach: Policy</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
<br />
== Policy ==<br />
<br />
At Mozilla, like at many other organizations, we rely on data to make product decisions. But here, unlike many other organizations, we balance our goal of collecting useful, high-quality data with our goal to give users meaningful choice and control over their own data. Here are some of our policies around this:<br />
<br />
* [[Data_Collection|Data Collection]]: Our policies around data collection<br />
* [[Data_Publishing|Data Publishing]]: How we publish (a subset of) our data for the public benefit<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233594Data2021-02-08T16:36:30Z<p>Wlach: more tweaks</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Data Documentation].<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry&diff=1233592Telemetry2021-02-08T15:54:11Z<p>Wlach: Redirect to Data</p>
<hr />
<div>#REDIRECT [[Data]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233591Data2021-02-08T15:53:23Z<p>Wlach: /* Resources */</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Firefox Data Documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233590Data2021-02-08T15:52:57Z<p>Wlach: Updates</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
'''Vision''': A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
'''Mission''': We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Resources ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
For detailed technical information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Firefox Data Documentation].<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233589Data2021-02-08T15:35:22Z<p>Wlach: </p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
== Get in touch ==<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233588Data2021-02-08T15:34:28Z<p>Wlach: /* Working groups */ 'WG'->''</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary]<br />
<br />
= Get in touch =<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/WorkingGroups/GleanDictionary&diff=1233587Data/WorkingGroups/GleanDictionary2021-02-08T15:33:24Z<p>Wlach: Public working group</p>
<hr />
<div>= Quick Reference =<br />
<br />
{| class="wikitable"<br />
|-<br />
| Development URL || [https://dictionary.protosaur.dev dictionary.protosaur.dev]<br />
|-<br />
| Source code || [https://github.com/mozilla/glean-dictionary mozilla/glean-dictionary on GitHub]<br />
|-<br />
| Matrix Channel || [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary on chat.mozilla.org]<br />
|-<br />
| Meetings || Fortnightly cadence on Wednesdays at [http://www.timebie.com/std/utc.php?q=17 5pm UTC].<br />
<br />
Zoom Room: [https://mozilla.zoom.us/j/91218334714 91218334714]<br />
<br />
[https://docs.google.com/document/d/1v7PV781CjmVNXxzlwzCNQvUT-AbpvE2n-cKe5gKI8zQ/edit Meeting Notes] open to the public<br />
|}<br />
<br />
= Charter =<br />
<br />
The focus of this working group is to produce a usable version of the Glean Dictionary, a [https://en.wikipedia.org/wiki/Data_dictionary data dictionary] for applications written using Glean SDK. This includes Firefox for Android and Firefox for iOS. As [https://firefox-source-docs.mozilla.org/toolkit/components/glean/index.html Firefox on Glean] comes together, the Glean Dictionary will also index the metadata sent by Firefox Desktop.<br />
<br />
There are currently four planned phases for the project:<br />
<br />
# (done) Index Glean metric and ping data produced by Glean applications, providing links to their representation in BigQuery<br />
# (in progress) Index derived datasets produced by [https://github.com/mozilla/bigquery-etl bigquery-etl], creating generated documentation at [https://mozilla.github.io/bigquery-etl bigquery-etl]<br />
# (not started) Represent and link the dataset documentation outlined above in the Glean Dictionary<br />
# (planning) Add facilities to annotate glean metrics and pings with additional data and commentary useful for data scientists and other data practitioners at Mozilla<br />
<br />
After phase 4 of the project is completed, we will re-evaluate the future of this group.<br />
<br />
= Communication =<br />
<br />
There are three primary communication channels for the group:<br />
<br />
* A fortnightly Zoom meeting open to all, for synchronous discussion and larger-scale strategic planning<br />
* The #glean-dictionary channel on Matrix for quick questions<br />
* Discussions in GitHub issues for design discussions and all other questions<br />
<br />
Generally, discussions in GitHub issues is preferred since it can happen asynchronously and leaves a historical record that we can refer back to. For larger changes, consider writing a proposal (see below).<br />
<br />
All project communications should follow the [https://www.mozilla.org/about/governance/policies/participation/ Mozilla Community Participation Guidelines].<br />
<br />
= Stakeholders = <br />
<br />
Glean team, Data science, the Data Taxonomy Effort, other consumers of data.<br />
<br />
= Membership =<br />
<br />
We welcome your feedback and involvement! We work in the open and anyone from the Mozilla community is welcome to join this group. This project involves a variety of pieces emphasizing different technologies, including:<br />
<br />
* Building out the frontend (JavaScript, svelte)<br />
* Working on the data infrastructure pieces to gather metadata (python, BigQuery)<br />
* Improving data documentation and metadata definitions through Mozilla (markdown, yaml, python)<br />
<br />
If you want to contribute, but aren't sure where to start, join our [https://chat.mozilla.org/#/room/#glean-dictionary:mozilla.org #glean-dictionary matrix channel] and say hi! Someone can probably find an initial task for you to work on.<br />
<br />
= Coordination =<br />
<br />
This group is currently being coordinated by Will Lachance (wlach on Github, Matrix, and Mozilla Slack), who is responsible for creating the meeting agenda and ensuring that discussions lead to a productive places. Feel free to get in touch if you have feedback or questions!<br />
<br />
= Proposals =<br />
<br />
{| class="wikitable"<br />
|-<br />
! Proposal !! Date !! Status<br />
|-<br />
| [https://docs.google.com/document/d/1cCFldtc7Ki5kbAlCeIVQ6lPbzWl_DAMr46lCUAWRWGg/edit# Working Group Proposal] || 20 November 2020 || Accepted<br />
|-<br />
| [https://docs.google.com/document/d/1OkTWA3rsSJ0m5g9GDnxXVUMkJP-xJMQk_bDgDq-Z9xM/edit# Initial Proposal] || 10 August 2020 || Accepted<br />
|-</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233583Data2021-02-08T14:39:05Z<p>Wlach: Glean Dictionary Working Group</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
* [https://wiki.mozilla.org/Data/WorkingGroups/GleanDictionary Glean Dictionary WG] <br />
<br />
= Get in touch =<br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233582Data2021-02-08T14:31:03Z<p>Wlach: tweaks / fixes</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learn more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see [https://telemetry.mozilla.org telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
== Get in touch == <br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization mana page] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233581Data2021-02-08T14:29:37Z<p>Wlach: update title</p>
<hr />
<div>= Data @ Mozilla =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learning more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see the [telemetry.mozilla.org portal https://telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
== Getting in touch == <br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [mana page https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1233580Data2021-02-08T14:29:10Z<p>Wlach: Revamp</p>
<hr />
<div>= Data =<br />
<br />
== Who are we? ==<br />
<br />
The Data team at Mozilla is made up of people with many different skills to facilitate more than just simple reporting: people who have a deep knowledge of data processing infrastructure, understanding the business and market, as well as statistical and modeling knowledge.<br />
<br />
== Vision ==<br />
<br />
A sustainable, global and independent Mozilla that ethically leverages data for the common good.<br />
<br />
== Mission ==<br />
<br />
We provide tools and guidance to transform Mozilla into a modern, global, ethical data-informed business.<br />
<br />
== Learning more ==<br />
<br />
If you’re a Mozilla community member (or are just curious) see the [telemetry.mozilla.org portal https://telemetry.mozilla.org], which has links to our documentation, resources, and dashboards -- many of which are open to the public.<br />
<br />
== Working groups ==<br />
<br />
The Data Organization has a number of working groups. Some of them are open to participation by the the community:<br />
<br />
== Getting in touch == <br />
<br />
We hang out on the [https://chat.mozilla.org/#/room/#telemetry:mozilla.org #telemetry] channel on chat.mozilla.org. Come say hi!<br />
<br />
If you are a Mozilla employee, more information on how to engage with us internally is on our [mana page https://mana.mozilla.org/wiki/display/DATA/Mozilla+Data+Organization] (LDAP access required).</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data_Publishing&diff=1233508Data Publishing2021-02-04T22:02:36Z<p>Wlach: Add table of contents, linkable headers</p>
<hr />
<div>== Introduction ==<br />
<br />
Mozilla’s history is steeped in openness and transparency - it’s simply core to what we do and how we see ourselves in the world. We are always looking for ways to bring our mission to life in ways that help create a healthy internet and support the Mozilla Manifesto. One of our commitments says “We are committed to an internet that elevates critical thinking, reasoned argument, shared knowledge, and verifiable facts”. <br />
<br />
To this end, we have spent a good amount of time considering how we can publicly share our Mozilla telemetry data sets - it is one of the most simple and effective ways we can enable collaboration and share knowledge. But, only if it can be done safely and in a privacy protecting, principled way. We believe we’ve designed a way to do this and we are excited to outline our approach here.<br />
<br />
== Dataset Publishing Process ==<br />
<br />
We want our data publishing review process, as well as our review decisions to be public and understandable, similar to our [[Firefox/Data_Collection|Mozilla Data Collection]] program. To that end, our full dataset publishing policy and details about what considerations we look at before determining what is safe to publish can be found below, including a summary of the critical pieces of that process.<br />
<br />
The goal of our data publishing process is to:<br />
<br />
* Reduce friction for data publishing requests with low privacy risk to users;<br />
* Have a review system of checks and balances that considers both data aggregations and data level sensitivities to determine privacy risk prior to publishing, and; <br />
* Create a public record of these reviews, including making data and the queries that generate it publicly available and putting a link to the dataset + metadata on a public-facing Mozilla property.<br />
<br />
This page defines all of the factors that must be taken into consideration before publicly sharing Mozilla’s telemetry data. It describes:<br />
<br />
* The levels of possible dataset aggregations using Mozilla’s data<br />
* The levels of publishing sensitivity<br />
* What dimensions are sensitive, and at which level<br />
* What metrics are sensitive, and at which level<br />
* How we characterize the levels of aggregation<br />
<br />
## How we characterize the levels of aggregation<br />
<br />
The table below describes the various types of aggregation levels we are defining.<br />
{| class="wikitable"<br />
|-<br />
! Level !! Aggregation !! Examples<br />
|-<br />
| 1 || '''Statistical / ML Models''' <br /> A model built/trained using real data. || TAAR, Federated learning models, Forecasting models<br />
|-<br />
| 2 || '''Dimension-level aggregation w/ minimum bucket sizes''' <br /> Aggregated by dimensions, minimum "bucket" size of population 5,000. || Total page loads by country, OS, locale, channel where any combination with a count less than 5,000 are grouped into “Other”<br />
[Canada, Linux, “Other locales”, nightly] for rare locales<br />
|-<br />
| 3 || '''Dimension-level aggregation w/o minimum bucket sizes''' <br /> Aggregated by dimensions, no minimum bucket size. || Client ID count by country, os, locale, channel, where there could be: [Canada, Linux, PL, nightly] which has one client in it.<br />
|-<br />
| 4 || '''Probabilistic Aggregates''' <br /> Data structures for approximations. || [https://en.wikipedia.org/wiki/HyperLogLog HLL] for computing approximate unique client counts, [https://en.wikipedia.org/wiki/Bloom_filter bloom filter] for computing presence in a set.<br />
|-<br />
| 5 || '''Anonymized individual-level data''' <br /> Covers “partial aggregates” like clients_daily which is aggregated by day. Key feature is that it still has an individual-level identifier. Actual identifiers are anonymized using a one-to-one replacement value. In this example, we replaced the ID with A, B, C, etc. || <br />
* Anonymized_id, date, country, os, locale, channel<br />
* A, 2019-08-08, Canada, Linux, PL, nightly<br />
* A, 2019-08-09, Canada, Linux, PL, nightly<br />
* A, 2019-08-10, Canada, Linux, PL, nightly<br />
* B, 2019-08-10, Peru, Windows, EN, release<br />
|-<br />
| 6 || '''Not-anonymized individual-level data''' <br /> This data contains individual-level identifiers as they exist in the raw data. Compared with anonymized data, instead of A, B, and C we use the original identifiers. || <br />
* actual_id, date, country, os, locale, channel<br />
* 859c8a32-0b73-b547-a5e7-8ef4ed9c4c2d, 2019-08-08, Canada, Linux, PL, nightly<br />
* 859c8a32-0b73-b547-a5e7-8ef4ed9c4c2d, 2019-08-09, Canada, Linux, PL, nightly<br />
* 859c8a32-0b73-b547-a5e7-8ef4ed9c4c2d, 2019-08-10, Canada, Linux, PL, nightly<br />
* 4db8d07d-1935-9c45-93c9-6d97a790bb12, 2019-08-10, Peru, Windows, EN, release<br />
|-<br />
| 7 || '''High resolution individual-level data''' <br /> The highest level of resolution is releasing events at the per-second or per-subsecond resolution. || Raw telemetry events data, a sequence of actions in order of occurrence.<br />
|-<br />
|}<br />
<br />
== How we characterize the sensitivity of dimensions ==<br />
<br />
Based on the [[https://wiki.mozilla.org/Firefox/Data_Collection#Data_Collection_Categories|Data Collection Categories]], most Telemetry data naturally falls within category 1 (technical data) and 2 (interaction data), which are not considered sensitive. A notable exception, however, is geo location, which we geocode from IP addresses to extract City / Region / Country, but only include cities with a population > 15,000 (according to the Geonames database). <br />
<br />
Category 3 (web activity) or 4 (highly-sensitive) data should be excluded from the set of “safe” dimensions.<br />
<br />
Matrix of aggregation safety vs. dimension sensitivity:<br />
{| class="wikitable"<br />
|-<br />
! Category !! Aggregation Level !! Notes<br />
|-<br />
| '''Category 1 (Technical) and 2 (Interaction)''' || '''1, 2, 3''' || For low-sensitivity data, we may not require a minimum bucket size for aggregation.<br />
|-<br />
| '''Category 3 (Web Activity)''' || '''1, 2''' || As sensitivity increases, minimum bucket size becomes increasingly important.<br />
|-<br />
| '''Category 4 (Highly Sensitive Data)''' || '''1, 2''' || Technically, category 4 often involves highly sensitive data, such as explicit identifiers, that will be removed in the process of aggregation. We include it here for the sake of completeness.<br />
|}<br />
== How do we characterize the sensitivity of metrics? ==<br />
<br />
Most metrics are not sensitive information, per se. That said, if a metric indicates or directly implies something about revenue, it is “sensitive”. Example: Search counts.<br />
<br />
== Dataset Publishing Process ==<br />
<br />
The goal of this process is to (1) make the “easy” (that is, safe) data publishing requests relatively friction-less, (2) have guard rails in-place so we don’t publish something that exposes us or our users to risk in some way, and (3) ensure that the dataset publishing request process matches closely other processes that are familiar to the data stewards.<br />
<br />
Having a dataset published requires filling out a bug. Requests will use the nomenclature defined in the preceding sections to answer a series of questions including the following four. If the answer to all of them is “no”, the data may be published. A “yes” above means extra review is required.<br />
<br />
* Is the level of aggregation 3 or higher?<br />
* Are there any Data Collection Category 3 (web activity) or 4 (highly-sensitive) dimensions?<br />
* Do any of the dimensions or metrics include sensitive data?<br />
* Are there any data included that do not have a corresponding data review for collection? Please link to relevant data review(s).<br />
<br />
A data steward will then be assigned to the bug, either by the requester or as part of bug triage, to double-check that these questions are correctly answered and there are no confounding factors inherent to the publishing of the data.<br />
<br />
Once the request is approved, data engineering will do the implementation work:<br />
* Write (or review) the query<br />
* Schedule it to update on the desired frequency<br />
* Plumb it in to the public facing dataset infrastructure, including metadata that links the public data back to the above review bug.<br />
* Once the dataset has been published, it will be announced on the [https://blog.mozilla.org/data/ Data @ Mozilla blog]. Accessing the public data is described on the [https://docs.telemetry.mozilla.org/cookbooks/public_data.html data documentation page].<br />
<br />
== Definitions ==<br />
<br />
'''Metric''' - A metric is anything we want to measure.<br />
Examples: the number of clients that used the developer tools console, the number of active clients.<br />
<br />
'''Dimension''' - A dimension is a qualitative value such as OS, channel, or date. In practice, a dimension often defines a sub-population on which we can calculate a metric, allowing us to segment the metric for further analysis.<br />
Examples: if we have an OS dimension, we can analyze the number of active clients by OS.<br />
<br />
'''Aggregate''' - A combined value of many measurements (metric values), typically grouped by dimension or sets of dimensions.<br />
<br />
'''Individual-level Data''' - Data containing a dimension which uniquely identifies a single profile, user, client, etc.<br />
<br />
'''Tabular Data''' - Data that consists of rows (or records) and columns (or fields). Each row has the same number of columns, and each column represents a dimension or metric for that row. Think of a spreadsheet or CSV file as examples of this type of data.<br />
<br />
== What's Been Published So Far? ==<br />
<br />
Our publicly available datasets are [https://public-data.telemetry.mozilla.org/all-datasets.json here].</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry&diff=1225510Telemetry2020-03-27T13:17:10Z<p>Wlach: Link to source docs</p>
<hr />
<div>It's helpful for Mozilla's engineers and decision-makers to be able to measure how Firefox behaves in the real world. The Telemetry feature provides this capability by sending performance and usage info to Mozilla. As you use Firefox, Telemetry measures and collects non-personal information, such as performance, hardware, usage and customizations. It then sends this information to Mozilla on a daily basis and we use it to improve Firefox. <br />
<br />
For detailed technical information on how this data is collected, as well as information on how to add Telemetry to Firefox, see [https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/index.html Telemetry reference in the Firefox source documentation].<br />
<br />
For more information on accessing and using this information in analysis, see the [https://docs.telemetry.mozilla.org/ Firefox Data Documentation].</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry&diff=1224116Telemetry2020-02-21T16:52:32Z<p>Wlach: remove unneeded title</p>
<hr />
<div>It's helpful for Mozilla's engineers and decision-makers to be able to measure how Firefox behaves in the real world. The Telemetry feature provides this capability by sending performance and usage info to Mozilla. As you use Firefox, Telemetry measures and collects non-personal information, such as performance, hardware, usage and customizations. It then sends this information to Mozilla on a daily basis and we use it to improve Firefox. <br />
<br />
For more information, see the [https://docs.telemetry.mozilla.org/ Firefox Data Documentation].</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry&diff=1224115Telemetry2020-02-21T16:52:07Z<p>Wlach: prune obsolete info, make this page into a simple redirect to docs.tmo for now</p>
<hr />
<div>=Telemetry=<br />
It's helpful for Mozilla's engineers and decision-makers to be able to measure how Firefox behaves in the real world. The Telemetry feature provides this capability by sending performance and usage info to Mozilla. As you use Firefox, Telemetry measures and collects non-personal information, such as performance, hardware, usage and customizations. It then sends this information to Mozilla on a daily basis and we use it to improve Firefox. <br />
<br />
For more information, see the [https://docs.telemetry.mozilla.org/ Firefox Data Documentation].</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data&diff=1224114Data2020-02-21T16:48:44Z<p>Wlach: this page is obsolete, redirect to telemetry</p>
<hr />
<div>#Redirect [[Telemetry]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=Data/Platform&diff=1224113Data/Platform2020-02-21T16:48:06Z<p>Wlach: this page is obsolete, redirect to telemetry</p>
<hr />
<div>#REDIRECT [[Telemetry]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=WeeklyUpdates/2019-09-16&diff=1217914WeeklyUpdates/2019-09-162019-09-16T17:09:54Z<p>Wlach: /* Welcome! */ welcome ben wu</p>
<hr />
<div><br />
{{WeeklyUpdateNav}}<br />
* Every Monday @ 11:00am Pacific Time (19:00 UTC) <br />
* Watch at https://mzl.la/project-meeting-2019-09-16 or anonymously at https://air.mozilla.org. You can also watch on the Mozilla YouTube channel at https://www.youtube.com/watch?v=OB6N8xRMe3k.<br />
* join #weekly-project-call on irc.mozilla.org or Slack for backchannel discussion<br />
* If you plan on presenting, please join the Zoom video chat 20 minutes prior to the start of the meeting and announce to the A/V Technicians that you will be speaking so that they can confirm your Audio and Video.<br />
* '''Presenters only:''' Zoom Meeting ID: 828 817 988 [https://mozilla.zoom.us/j/828817988 https://mozilla.zoom.us/j/828817988.] Do '''not''' use this room if you're not planning to speak. <br />
* One-tap mobile<br />
** +16465588656,,828817988# US (New York)<br />
** +17207072699,,828817988# US<br />
* Dial-in by your location<br />
** +1 646 558 8656 US (New York)<br />
** +1 720 707 2699 US<br />
** 877 853 5257 US Toll-free<br />
** +61 2 8015 2088 Australia<br />
** +61 8 7150 1149 Australia<br />
** 1800 893 423 Australia Toll-free<br />
** +1 647 558 0588 Canada<br />
** +33 1 8288 0188 France<br />
** +33 7 5678 4048 France<br />
** 0 805 082 588 France Toll-free<br />
** +49 30 5679 5800 Germany<br />
** +49 69 8088 3899 Germany<br />
** +49 30 3080 6188 Germany<br />
** 800 724 3138 Germany Toll-free<br />
** +852 5808 6088 Hong Kong, China<br />
** +44 203 695 0088 United Kingdom<br />
** +44 203 966 3809 United Kingdom<br />
** +44 203 051 2874 United Kingdom<br />
** 0 800 031 5717 United Kingdom Toll-free<br />
<br />
__TOC__<br />
<br />
= Weekly Project Status Meeting Agenda =<br />
<br />
Items in this section will be shared during the live status meeting.<br />
<br />
== Friends of Mozilla [[Image:Tree.gif|Friends of Mozilla]] ==<br />
<br />
== Upcoming Events ==<br />
<br />
=== This Week ===<br />
<br />
=== Monday, {{#time:d F|{{SUBPAGENAME}}}} ===<br />
<br />
=== Tuesday, {{#time:d F|{{SUBPAGENAME}} +1 day}} ===<br />
<br />
=== Wednesday, {{#time:d F|{{SUBPAGENAME}} +2 days}} ===<br />
* Homebrew Website Club <br />
** The '''Homebrew Website Club''' is a growing world-wide network of meetups for everyone who wants to take back their web experience from social media silos, and own their online identities, content, and interactions.<br />
<br />
* 17:30-19:30 (local times) at<br />
** '''Nürnberg (GERMANY)'''<br />
** '''Karlsruhe (GERMANY)'''<br />
** '''London (ENGLAND)'''<br />
** '''Nottingham (ENGLAND)'''<br />
** '''Seattle (WA)'''<br />
* Details and '''RSVP''': https://indieweb.org/events/2019-09-18-homebrew-website-club<br />
<br />
=== Thursday, {{#time:d F|{{SUBPAGENAME}} +3 days}} ===<br />
* '''Homebrew Website Club'''<br />
** '''Brighton (ENGLAND)''' (every Thursday)<br />
** '''Oakland (CA)''' {{newbadge}}<br />
* Details and '''RSVP''': https://indieweb.org/events/2019-09-19-homebrew-website-club<br />
<br />
* [https://docs.google.com/document/d/1i59ph_aEKLWitF8BFrnIZPZTLuaYnwZEi1qxeyhKP0g/edit# Add-ons Show & Tell]<br />
** Bring your add-ons related projects to show and tell! <br />
** 14:30 UTC / 7:30am PST <br />
** Sign ups and call-in info are on the [https://docs.google.com/document/d/1i59ph_aEKLWitF8BFrnIZPZTLuaYnwZEi1qxeyhKP0g/edit# agenda].<br />
<br />
=== Friday, {{#time:d F|{{SUBPAGENAME}} +4 days}} ===<br />
<br />
=== Saturday, {{#time:d F|{{SUBPAGENAME}} +5 days}} ===<br />
<br />
=== Sunday, {{#time:d F|{{SUBPAGENAME}} +6 days}} ===<br />
<br />
=== Next Week ===<br />
<br />
=== Later Events and Available Tickets ===<br />
<br />
'''Mozilla DevRel Complimentary Tickets'''<br />
<br />
Our DevRel Sponsorship team sometimes receives complimentary tickets to share with Mozillians interested in attending the following events. Please fill out this form [https://airtable.com/shrRFuIEm7j2a0Gtu Ticket RequestForm ]. <br />
If you have questions, reach out to the DevSponsorship Team at devsponsorship@mozilla.com for more details. <br />
<br />
'''[https://jsconfbp.com/ JS Conf & CSS Conf Budapest]''' Budapest, Hungary | 2019-09-25 & 2019-09-26-27<br />
<br />
'''[https://fitc.ca/event/webu19/ Web Unleashed 2019]''' Toronto, Canada | 2019-09-13-14<br />
<br />
'''[https://redevelop.io/ redevelop]''' Bournemouth, England | 2019-20-09<br />
<br />
'''[https://ruhrjs.de/ RuhrJS 2019]''' Rotunde Bochum - Bochum Germany | 2019-10-05 to 2019-10-06<br />
<br />
'''[http://connect.tech/ CONNECT.TECH]''' Atlanta, GA, USA | 2019-10-16 to 2019-10-18<br />
<br />
'''[https://2018.ffconf.org/ ffconf]''' ffconf (formerly known as Full Frontal) Brighton, England | 2019-10-08 to 2019-10-08<br />
<br />
'''[https://scriptconf.org/ Script'19: Javascript for Everyone]'''| Linz, Austria | 2019-10-25<br />
<br />
'''[https://perfnow.nl/ performance.now ()]'''| Amsterdam, The Netherlands | 2019-11-21 to 2019-11-22<br />
<br />
'''[https://halfstackconf.com/ Halfstack]''' London | 2019-11-22 <br />
<br />
'''[https://2019.webconf.asia/ Webconf Asia]''' Hong Kong | 2019-11-20-23 <br />
<br />
'''[https://ntnu.edu/wac2019 Web Audio Conference 2019]''' Trondheim, Norway | 2019-12-04 to 2019-12-06<br />
<br />
'''[https://halfstackconf.com/ Halfstack]''' Phoenix, AZ | 2020-01-17 <br />
<br />
<br />
''' Mozilla Sponsored Developer Events'''<br />
<br />
Current list of Mozilla sponsored developer events (with their start dates) around the globe (DevRel Events team will update):<br />
<br />
Silicon Slopes Localization - Cross-Cultural Management | 9/17/19| Salt Lake City<br />
<br />
Halfstack | 9/19/19| Vienna<br />
<br />
Armada JS | 9-19/19 | Novi Sad, Serbia<br />
<br />
re:develop | 9/20/19 | Bournemouth, UK<br />
<br />
We Love Speed | 9/20/19 | Lille, France<br />
<br />
Web Weekend Kathmandu | 9/21-22/19 | Kathmandu<br />
<br />
Finch Front End | 9/23-25/19 | Edinburgh, UK<br />
<br />
CSSConf Budapest 2019 | 9/25/19 | Budapest, Hungary<br />
<br />
JSConf Budapest 2019 | 9/26-27/19 | Budapest, Hungary<br />
<br />
PromptConf | 09/28/19 | Chicago<br />
<br />
ARTIFACT Conference | 9/30-10/2 19 | Austin, Texas US<br />
<br />
[https://indieweb.org/2019/Amsterdam ❌ IndieWebCamp Amsterdam] | 9/28/19 - 9/29/19 | Amsterdam, NL | 9/28/19 - 9/29/19 | Amsterdam, NL<br />
<br />
View Source 2019 | 9/30/19 - 10/2/19 | Amsterdam, NL<br />
<br />
Fronteers | 10/3/19 | Amsterdam, NL<br />
<br />
RuhrJS 2019 | 10/5/19 | Bochum, Germany<br />
<br />
[https://indieweb.org/2019/NYC 🗽 IndieWebCamp New York City 2019] | 10/5/19 - 10/6/19 | New York City, NY, USA<br />
<br />
Paris Web 2019 | 10/10-12/19 | Bois-Colombes and Paris, France<br />
<br />
Web Engines Hackfest 2019 | 10/14/19 | Galicia, Spain<br />
<br />
CONNECT.TECH | 10/16/19 | Atlanta, GA, USA<br />
<br />
[https://indieweb.org/2019/Brighton 🎪 IndieWebCamp Brighton 2019] | 10/19/19 - 10/20/19 | Brighton, England<br />
<br />
2019 LLVM Developers' Meeting | 10/22/19 | San Jose<br />
<br />
a11yTO Conf | 10/24/19 | Toronto<br />
<br />
Script'19: Javascript for Everyone 10/25/19 | Linz, Austria<br />
<br />
Security BSides Portland | 10/25-26/19 | Portland, USA<br />
<br />
beyond tellerrand // | BERLIN 2019 | 11/4/19 | Berlin, Germany<br />
<br />
CascadiaJS | 11/07/19 | Seattle<br />
<br />
ffconf | 11/8/19 | Brighton, UK<br />
<br />
RustFest Barcelona | 11/09/19 | Barcelona<br />
<br />
Performance now |11/21-22/19 | Amsterdam, The Netherlands<br />
<br />
Halfstack | 11/22/19| London<br />
<br />
WebConf Asia | 11/20-23| Hong Kong<br />
<br />
webclerks community conference | 11/25/19 | Vienna<br />
<br />
Web Audio Conference 2019 | 12/4/19 | Trondheim, Norway<br />
<br />
dotJS 2019 | 12/5-6/19<br />
<br />
Halfstack | 01/17/20 | Phoenix<br />
<br />
JS Kongress Munich | 04/15-16/20 | Munich, Germany<br />
<br />
[https://indieweb.org/2020 ⛰ IndieWeb Summit 2020] | 06/27/20 - 06/28/20 | Portland, Oregon, USA<br />
<br />
== Speakers ==<br />
<br />
The limit is '''3 minutes per topic'''. It's like a lightning talk, but don't feel that you have to have slides in order to make a presentation. If you plan on showing a video, you need to contact the Air Mozilla team before the day of the meeting or you will be deferred to the next week. The meeting is streamed in a 4:3 format in order to allow for split screen. If your slides are 16:9 "widescreen" format, please indicate in the "Sharing" column below.<br />
<br />
{| class="fullwidth-table wikitable"<br />
|-<br />
! [https://mozillians.org/u/USERNAME Presenter]<br />
! Title<br />
! Topic<br />
! Location<br />
! Sharing<br />
! Media<br />
! More Details<br />
|-<br />
| Who Are You?<br />
| What Do You Do?<br />
| What are you going to talk about?<br />
| Where are you presenting from? (Moz Space, your house, space)<br />
| Will you be sharing your screen? (yes/no, 4:3 or 16:9)<br />
| Links to slides or images you want displayed on screen<br />
| Link to where audience can find out more information<br />
|-<br />
| Elgin-Skye McLaren<br />
| Hubs Community & Events Manager<br />
| Emerging Technologies weekly update<br />
| Hubs Commons<br />
| yes<br />
| n/a<br />
| [https://wiki.mozilla.org/WeeklyUpdates/EmergingTechnology#September_16th.2C_2019 ET headlines]<br />
|-<br />
| Janice Wait + Chris Lawrence <br />
| Program Officer, Mozilla Foundation and Director of Strategy, Loup Design and Innovation<br />
| Grant for the Web<br />
| remote<br />
| no<br />
| n/a<br />
| [https://foundation.mozilla.org/en/blog/100-million-investment-reshape-economics-web/ Grant for the Web]<br />
|-<br />
|}<br />
<br />
= Welcome! =<br />
<br />
Let's say hello to some new Mozillians! If you are not able to join the meeting live, you can add a link to a short video introducing yourself.<br />
<br />
{| class="fullwidth-table wikitable"<br />
|-<br />
! ''Who is being introduced?''<br />
! ''Who are you? (the introducer)''<br />
! ''Where are you doing the introduction?''<br />
! ''Where are they from?''<br />
! ''How will they be part of Mozilla?''<br />
|-<br />
<!-- Insert new rows here --><br />
|-<br />
| Benjamin Wu<br />
| William Lachance<br />
| Toronto office<br />
| Toronto office<br />
| Data platform engineer<br />
|-<br />
|}<br />
<br />
<br />
[[Category:Weekly Updates]]<br />
[[Category:Meeting Notes]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=Firefox/Channels/Meetings/2018-05-15&diff=1193889Firefox/Channels/Meetings/2018-05-152018-05-15T15:52:10Z<p>Wlach: /* Nightly */</p>
<hr />
<div>'''Channel Meeting Details'''<br />
* Tuesdays 10am and Thursdays at 8am Pacific Time<br />
* [irc://irc.mozilla.org/planning irc.mozilla.org #planning] for backchannel<br />
<br />
'''Video/Teleconference Details - NEW'''<br />
* 650-903-0800 or 650-215-1282 x92 Conf# '''99951''' (US/INTL)<br />
* 1-800-707-2533 (pin 369) Conf# '''99951''' (US)<br />
* Vidyo Room: '''Release Coordination'''<br />
* Vidyo [https://v.mozilla.com/flex.html?roomdirect.html&key=EXysadF68Dac Guest URL]<br />
<br />
{{Notes}}<br />
<br />
__FORCETOC__<br />
<br />
<br />
== Attendees ==<br />
<br />
<br />
== Previous Actions ==<br />
<br />
== Schedule Update ==<br />
<br />
==Add-ons==<br />
<br />
==Shield studies==<br />
* FYI: Shield studies in flight. https://trello.com/b/maMI2Fn6/shield-studies<br />
<br />
== Stability ==<br />
* Will should have an update on the Mission Control Work<br />
<br />
===Nightly===<br />
awsy rate: 1.54 (browser: 0.79, content: 0.76) <br />
<br />
m-c nightly summary score: 9.14 (= √4(3.09 windows * 4.83 mac * 8.54 linux * 54.84 android)) - https://data-missioncontrol.dev.mozaws.net/#/?channel=nightly<br />
* windows (content_crashes: 1.89 / main_crashes: 1.2)<br />
* mac (content_crashes: 0.75 / main_crashes: 4.08)<br />
* linux (content_crashes: 3.65 / main_crashes: 4.89)<br />
* android (main_crashes: 54.84)<br />
<br />
[marcia]<br />
* New Service Workers regression https://bugzilla.mozilla.org/show_bug.cgi?id=1461667<br />
* Layout|Web Painting https://bugzilla.mozilla.org/show_bug.cgi?id=1461267 filed, but might be a dupe of another bug.<br />
* Nightly crash triage has been quiet the past few days - https://wiki.mozilla.org/NightlyCrashTriage/2018Q2<br />
<br />
===Beta===<br />
awsy rate: 0.79 (browser: 0.44, content: 0.36) <br />
<br />
===Release===<br />
awsy rate: 0.65 (browser: 0.38, content: 0.27) <br />
<br />
===Mobile===<br />
awsy rate: release 0.45 - beta 0.65 - nightly 1.55 <br />
<br />
*Release<br />
**No new crashes in the top 30 list<br />
**#4 top JS crash has some discussion ongoing - https://bugzilla.mozilla.org/show_bug.cgi?id=1461480<br />
<br />
*Beta<br />
** b3 data seems a bit low - 165 crashes total. Awaiting more data for b5.<br />
<br />
*Nightly<br />
**Top crash https://bugzilla.mozilla.org/show_bug.cgi?id=1460282 is verified fixed in 62.<br />
<br />
<br />
== Performance (Thu) ==<br />
=== Aurora / Dev Edition ===<br />
=== Beta ===<br />
=== Release ===<br />
== QE ==<br />
<br />
=== Nightly ===<br />
=== Aurora / Dev Edition ===<br />
=== Beta ===<br />
=== Action items ===<br />
=== Mobile ===<br />
<br />
==Performance==<br />
<br />
== RelEng ==<br />
<br />
== Marketing/Communications ==<br />
<br />
== User Advocacy ==<br />
<br />
== Roundtable ==<br />
<br />
<br />
== Special Topics ==<br />
=== Aurora/Beta Feature Review ===<br />
<br />
=== Post-Mortem (Tues 2wks from GA Release)===<br />
<br />
=== Sign Off (Thurs prior to GA Release) ===<br />
<br />
== Actions ==<br />
<br />
<br />
<br />
[[category:Channel_Meetings|{{SUBPAGENAME}}]]<br />
[[Category:Wikipage templates]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=Firefox/Channels/Meetings/2018-05-15&diff=1193888Firefox/Channels/Meetings/2018-05-152018-05-15T15:51:24Z<p>Wlach: /* Nightly */ nightly m-c rates</p>
<hr />
<div>'''Channel Meeting Details'''<br />
* Tuesdays 10am and Thursdays at 8am Pacific Time<br />
* [irc://irc.mozilla.org/planning irc.mozilla.org #planning] for backchannel<br />
<br />
'''Video/Teleconference Details - NEW'''<br />
* 650-903-0800 or 650-215-1282 x92 Conf# '''99951''' (US/INTL)<br />
* 1-800-707-2533 (pin 369) Conf# '''99951''' (US)<br />
* Vidyo Room: '''Release Coordination'''<br />
* Vidyo [https://v.mozilla.com/flex.html?roomdirect.html&key=EXysadF68Dac Guest URL]<br />
<br />
{{Notes}}<br />
<br />
__FORCETOC__<br />
<br />
<br />
== Attendees ==<br />
<br />
<br />
== Previous Actions ==<br />
<br />
== Schedule Update ==<br />
<br />
==Add-ons==<br />
<br />
==Shield studies==<br />
* FYI: Shield studies in flight. https://trello.com/b/maMI2Fn6/shield-studies<br />
<br />
== Stability ==<br />
* Will should have an update on the Mission Control Work<br />
<br />
===Nightly===<br />
awsy rate: 1.54 (browser: 0.79, content: 0.76) <br />
m-c nightly summary score: 9.14 (= √4(3.09 windows * 4.83 mac * 8.54 linux * 54.84 android)) - https://data-missioncontrol.dev.mozaws.net/#/?channel=nightly<br />
* windows (content_crashes: 1.89 / main_crashes: 1.2)<br />
* mac (content_crashes: 0.75 / main_crashes: 4.08)<br />
* linux (content_crashes: 3.65 / main_crashes: 4.89)<br />
* android (main_crashes: 54.84)<br />
<br />
[marcia]<br />
* New Service Workers regression https://bugzilla.mozilla.org/show_bug.cgi?id=1461667<br />
* Layout|Web Painting https://bugzilla.mozilla.org/show_bug.cgi?id=1461267 filed, but might be a dupe of another bug.<br />
* Nightly crash triage has been quiet the past few days - https://wiki.mozilla.org/NightlyCrashTriage/2018Q2<br />
<br />
===Beta===<br />
awsy rate: 0.79 (browser: 0.44, content: 0.36) <br />
<br />
===Release===<br />
awsy rate: 0.65 (browser: 0.38, content: 0.27) <br />
<br />
===Mobile===<br />
awsy rate: release 0.45 - beta 0.65 - nightly 1.55 <br />
<br />
*Release<br />
**No new crashes in the top 30 list<br />
**#4 top JS crash has some discussion ongoing - https://bugzilla.mozilla.org/show_bug.cgi?id=1461480<br />
<br />
*Beta<br />
** b3 data seems a bit low - 165 crashes total. Awaiting more data for b5.<br />
<br />
*Nightly<br />
**Top crash https://bugzilla.mozilla.org/show_bug.cgi?id=1460282 is verified fixed in 62.<br />
<br />
<br />
== Performance (Thu) ==<br />
=== Aurora / Dev Edition ===<br />
=== Beta ===<br />
=== Release ===<br />
== QE ==<br />
<br />
=== Nightly ===<br />
=== Aurora / Dev Edition ===<br />
=== Beta ===<br />
=== Action items ===<br />
=== Mobile ===<br />
<br />
==Performance==<br />
<br />
== RelEng ==<br />
<br />
== Marketing/Communications ==<br />
<br />
== User Advocacy ==<br />
<br />
== Roundtable ==<br />
<br />
<br />
== Special Topics ==<br />
=== Aurora/Beta Feature Review ===<br />
<br />
=== Post-Mortem (Tues 2wks from GA Release)===<br />
<br />
=== Sign Off (Thurs prior to GA Release) ===<br />
<br />
== Actions ==<br />
<br />
<br />
<br />
[[category:Channel_Meetings|{{SUBPAGENAME}}]]<br />
[[Category:Wikipage templates]]</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode18&diff=1190215Community:SummerOfCode182018-03-12T18:15:39Z<p>Wlach: fix link for metricsgraphics</p>
<hr />
<div>This is Mozilla's list of green-lit project proposals for the 2018 Google Summer of Code. <br />
<br />
<b>Are you a student looking to apply to GSoC with Mozilla?</b> You're in the right place. This page lists all the confirmmed [[SummerOfCode|Google Summer of Code]] projects. New suggestions can be made on [[Community:SummerOfCode18:Brainstorming|the Brainstorming page]]. Do not edit this page yourself; contact Mike Hoye or Florian for edits.<br />
<br />
If you're interested in participating in Mozilla's GSoC program, you can choose from the list below, '''but you do not have to'''. You can submit a proposal for your own idea. You should look at the [[Community:SummerOfCode18:Brainstorming|guidelines]], though, and discuss your ideas or application in the #introduction channel on IRC.Mozilla.org. This is important, as GSoC projects '''must have''' a supporting member of the Mozilla community to evaluate and mentor them, named in the application.<br />
<br />
===Application Advice===<br />
<br />
You should do the following:<br />
<br />
* Talk to the mentor. Contact details are on this page; if all you have is a nickname, reach out to them on IRC.<br />
* Read the [https://flossmanuals.net/GSoCStudentGuide/ GSoC Student Guide] and follow its advice.<br />
* Read [http://blog.gerv.net/2006/05/how_not_to_apply_for_summer_of/ How Not To Apply For Summer Of Code] and avoid doing the things listed there.<br />
* Read our examples of good applications: [[SummerOfCode/SampleApplications/1|1]], [[SummerOfCode/SampleApplications/2|2]], [[SummerOfCode/SampleApplications/3|3]].<br />
* Apply on [https://summerofcode.withgoogle.com/ the GSoC site] (note that we have an [[SummerOfCode/ApplicationTemplate|application template]]).<br />
* It is entirely acceptable to apply for 2 or 3 projects, if more than one catches your eye; if the applications are high quality, that can improve your chances. Applying to more than that will seem like spam.<br />
<br />
Questions about individual projects are best addressed to the potential mentor of that project. These should be listed in the table below. If you want to contact a mentor and contact details are not here, ask people in the #introduction channel on IRC: irc://irc.mozilla.org/#introduction. If you have questions of any other sort, send mail to [mailto:mhoye@mozilla.com Mike Hoye]. He will try and respond promptly and direct your questions to the right person. <br />
<br />
===Project List===<br />
<br />
* The following projects have been tentatively greenlit, pending milestone clarifications.<br />
<br />
== 2018 Project List ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Additional Comments<br />
|-<br />
| ReSpec <br />
| [https://github.com/w3c/respec/ ReSpec] is a JS-based tool used to write W3C Specifications (Web Standards) that is widely used by the Web Standards Community. With 5+ years of development, it's heavily depended upon by the W3C community at large (of which Mozilla is an active participant). ReSpec's code is in need of some modernization, optimizations, and bug fixes - and we could use your help! In this project, you would have the opportunity to make ReSpec's UI more accessible, making it leaner and faster using distributed processing with Web Workers, and/or adding new features to make the lives of W3C spec Editor's better. <br />
| Languages or skills needed: JavaScript, HTML, CSS. <br />
| [https://mozillians.org/en-US/u/mcaceres/ Marcos Caceres]<br />
| [https://mozillians.org/en-US/u/mcaceres/ Marcos Caceres]<br />
| [https://github.com/w3c/respec/issues/1502 Ideas list]. ReSpec offers students the opportunity to work on a large code base that has extensive real world use and impact. The project offers students an extensive range of problems to tackle, from UI design, to concurrent processing (using Web Workers to do distributed text processing), dealing with accessibility and internationalization, writing and learning about unit and integration tests, security, code review, etc. - as well as exposure to the W3C and the web standards community, this project also aims at teaching students about how web standards are put together. To determine if this is a project you would like to be part of, see the [https://github.com/w3c/respec/issues/ list of issues] you could work on. It's a great opportunity to learn about all aspects of open source software development, but with the freedom to take on small to large challenges over the Summer (depending on your skill level and level of confidence). About the mentor: Marcos Caceres is a Staff Engineer at Mozilla who has been working on Web Standards for over a decade. Marcos is the lead maintainer of ReSpec. Marcos has extensive experience mentoring developers and has previously successfully mentor a GSO student. <br />
|-<br />
| D3D11 backend for gfx-rs HAL<br />
| [https://github.com/gfx-rs/gfx gfx-rs] is a graphics abstraction library written in Rust and currently used for prototyping and investigation of [https://www.khronos.org/blog/khronos-announces-the-vulkan-portability-initiative Vulkan Portability] and [https://www.w3.org/community/gpu/ WebGPU] by Mozilla. The Hardware Abstraction Layer (HAL) of gfx-rs currently supports Vulkan, D3D12, Metal, and OpenGL. We want it to be powering [https://github.com/servo/webrender/ WebRender] for Firefox Quantum and Servo, and since WebRender currently runs on D3D11 (through [https://chromium.googlesource.com/angle/angle/+/master/README.md Angle]), we need to provide a native D3D11 backend. The implementation can be based off the existing D3D11 code in [https://github.com/gfx-rs/gfx/tree/pre-ll/src/backend/dx11 gfx-rs pre-HAL], and the main challenge is porting the code, optimizing, and ensuring it can successfully run the reference test suite.<br />
| Candidates should be familiar with Rust as well as low-level graphics development. Not having D3D11-specific experience is acceptable, given the willingness to learn it and a properly setup Windows development environment.<br />
| Dzmitry Malyshau (Mozilla)<br />
| Dzmitry Malyshau (Mozilla)<br />
| Proposal Timeline:<br />
* Before April 20: setup the Windows environment for DirectX development; run the examples and reference tests with existing Vulkan/DX12/GL backends; get in contact with developers on [https://gitter.im/gfx-rs/gfx Gitter].<br />
* April 20 – May 23: get familiar with gfx-rs HAL architecture, attempt to fix small entry-level issues and/or write an application/library on top of it.<br />
* May 23 – May 30: setup the backend skeleton with bits of actual logic, such as device initialization, and have it building on CI.<br />
* June 1 – July 5: implement the DX11 logic necessary to run the `quad` example and pass the basic graphics reference tests.<br />
* July 6 - July 31: implement support for compute and tessellation; benchmark the new backend against DX12; pass the full reference test suite.<br />
* Aug 1 - Aug 16: determine the list of capabilities that need to be exposed, merge it with GL backend legacy capabilities, expose to the reference tests.<br />
|-<br />
| Servo: Prototype ways of splitting up the script crate<br />
| [https://github.com/servo/servo/ Servo] is a web rendering engine written in Rust. One specific module contains a huge amount of code that is expensive to compile all at once. This project is intended to explore ways of splitting it into separate modules to achieve better build performance without breaking the complex set of interdependencies that exist in the code.<br />
| Candidates should be familiar with Rust, in particular using it in projects made of multiple crates and relying on features like traits and associated types.<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://github.com/servo/servo/wiki/Prototype-ways-of-splitting-the-script-crate Full project description page]<br />
|-<br />
| web-platform-tests: Improve test manifest workflow and performance<br />
| [https://github.com/w3c/web-platform-tests/ web-platform-tests] is a project for writing a cross-browser test suite, which is used across all major browser engines. The list of tests is stored in a JSON manifest file which is slow to generate and which we currently check in to the Firefox repo. These things create a poor developer experience. The goal of the project is to move to a system where the manifest is downloaded on demand for Firefox developers, and then to explore ways to speed up manifest generation such as employing parallelism or moving performance hotspots to Rust. <br />
| Candidates should be confident programming in Python.<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
| [https://github.com/jgraham/web-platform-tests/wiki/Improve-Test-Manifest-Workflow-and-Performance Full project description page]<br />
|-<br />
| WasmFiddle<br />
| [https://wasdk.github.io/WasmFiddle/ WasmFiddle] is a web application that facilitates learning of the WebAssembly and its usage in the browser. The goal is to create a learning tool that will allow user to quickly start a project that uses WebAssembly, learn based on examples, explore existing popular toolchains (emscripten, rust, etc), etc. The project will be similar to what jsfiddle, jsbin, or codepen proposes, but with additional elements specific to WebAssembly only, e.g. a code generation from C/C++ languages or how the code will be compiled by the browser and run in native machine code.<br />
| Languages or skills needed: JavaScript, HTML, CSS, WebAssembly. Suggested additional technologies: React, Rust, C++.<br />
| [mailto:ydelendik@mozilla.com Yury Delendik] :yury (Mozilla)<br />
| [mailto:ydelendik@mozilla.com Yury Delendik] :yury (Mozilla)<br />
| The project involves extending the functionality of the existing WasmFiddle/WasmExplorer projects to:<br />
* control, inspect and understand every part of the WebAssembly compilation pipeline;<br />
* don’t hide details or add magic build steps;<br />
* users can create, modify, fork and share fiddles, to reduce friction, no accounts;<br />
* influence JavaScript developers through well-thought out examples, best practices and templates;<br />
* add support for many languages: C/C++/Rust/etc;<br />
* create environments where users can play and have fun with: WebGL, Physics, Games, Audio Synthesizers.<br />
See [https://github.com/wasdk/WebAssemblyStudio/ WebAssemblyStudio repository] for new version of the WasmFiddle. <br />
|-<br />
| Pontoon's path to first contribution<br />
| Pontoon is Mozilla's localization tool, used by hundreds of contributors to translate Firefox, mozilla.org, and many other Mozilla products. It is critical to shipping Firefox to as many users as possible, all around the world. This year, we would like to improve the experience of first-time users of Pontoon. Your task would consist of defining a "path to first contribution", and making it as smooth as possible. That mainly involves adding a tutorial to walk users through the translation interface and reworking the landing page, all of that using recent front-end technologies in a fast-evolving application.<br />
| JavaScript, HTML, CSS, Python, UX. Django and React are pluses<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
|-<br />
| AOM - accessibility API<br />
| [https://github.com/WICG/aom/blob/gh-pages/explainer.md AOM] is JavaScript API designed to help the authors to make the web pages accessible, i.e. make them usable by the assisstive technologies such as screen readers, screen magnifiers and others. AOM specification is inspired by [https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA ARIA] standard, an HTML 5 extension, used to add semantics to the web pages. If you are familiar with ARIA, then you can think of AOM as JavaScript ARIA. The AOM proposal (phase 1) was already implemented in Chrome under a flag, and you can help do the same in Firefox.<br />
| Languages or skills needed: C++. <br />
| Alexander Surkov (Mozilla) (surkov.alexander@gmail.com)<br />
| Alexander Surkov (Mozilla) (surkov.alexander@gmail.com)<br />
| This project is the opportunity to hack on the heart of Mozilla Firefox, the Gecko engine. It involves working with a large code base and solving a wide variety of problems, from the solution architecture to finding efficient ways to implement it. If you want to learn about Gecko and accessibility, that's the right place. AOM has the potential of becoming a new standard in making the web page accessible next few years. You can help to happen it sooner.<br />
|-<br />
| metricsgraphics - zooming and brushing<br />
| [https://github.com/mozilla/metricsgraphics metricsgraphics] is JavaScript library designed to help with the presentation of interactive visualization. It is used extensively both inside and outside Mozilla. This project would be about adding a set of zooming and brushing features to facilitate the exploration of large and/or noisy datasets, such as in the [https://wiki.mozilla.org/EngineeringProductivity/Projects/Perfherder Perfherder] or [https://github.com/mozilla/missioncontrol/ Mission Control] projects. <br />
| Languages or skills needed: JavaScript, HTML/SVG<br />
| William Lachance (Mozilla)<br />
| William Lachance (Mozilla)<br />
| [https://docs.google.com/document/d/1_KIOJtemqlCBktDdfdjDuS4XhICeaKO3QhNRwbnnf-g/edit?usp=sharing Detailed project outline]<br />
|-<br />
| Iodide Notebook<br />
| The [https://github.com/mozilla/iodide Iodide Notebook] is a pure client-side browser-based IDE and publication tool for scientific computing and computational inquiry using web technologies (HTML, CSS, Javascript, WebAssembly). It provides a Jupyter-style notebook interface, granting scientists full access to the power of the DOM and browser APIs within a familiar iterative workflow, and with zero overhead -- absolutely no software installation and configuration is required, so people can seamlessly learn from and extend the work of others.<br />
<br />
Students will contribute to: extending the functionality of the notebook; making the notebook code more robust and performant; helping to shape the user experience; and creating example notebooks.<br />
| JS; HTML; CSS; React/Redux; familiarity with at least one scientific computing language preferred (ex: MATLAB, Mathematica, Julia, Numpy/Scipy, R, etc). Science/math background is a major plus.<br />
| Brendan Colloran (bcolloran@mozilla.com)<br />
| Brendan Colloran (bcolloran@mozilla.com)<br />
| In addition to helping to build the notebook, actively dogfooding it will be an essential part of our work. This means that students with a scientific and/or applied math background will be encouraged to build example notebooks that demonstrate visualizations, simulations, data analyses, etc., on topics of their choosing. In addition to mentoring students on software development, we will provide mentorship in scientific computing and data science.<br />
|<br />
|-<br />
| Improved Pulse Inspector Backend<br />
| [https://github.com/taskcluster/taskcluster-rfcs/issues/104 More details here]. The idea is to create a backend service that will allow browser-side applications like [https://tools.taskcluster.net/pulse-inspector/ pulse inspector] to listen to arbitrary [http://pulseguardian.mozilla.org/ Pulse] messages.<br />
| JavaScript (server-side), AMQP, WebSockets<br />
| Dustin Mitchell (Mozilla)<br />
| Jonas Jensen (Mozilla) / Eli (Perelman Mozilla)<br />
| This project can largely be a generic read-only websocket proxy for RabbitMQ, refer to our [https://github.com/taskcluster/taskcluster-rfcs/issues/104 issue for more details].<br />
|-<br />
| Redash - Improve user experience of [https://sql.telemetry.mozilla.org STMO]<br />
| Mozilla runs an [https://sql.telemetry.mozilla.org/ instance] of the Open Source project [https://github.com/getredash/redash Redash], a data analysis and dashboarding tool that has been customized and configured for use with a number of the Firefox organization's data sets. It provides low barrier access to the vast amount of Firefox Telemetry data (and others) to make product decisions for Firefox and others projects.<br />
| Languages or skills needed: Python, JavaScript, HTML<br />
| [https://mozillians.org/en-US/u/jezdez/ Jannis Leidel (jezdez)] (Mozilla)<br />
| [https://mozillians.org/en-US/u/jezdez/ Jannis Leidel (jezdez)] (Mozilla)<br />
| Knowing SQL or having experience with data analysis is not a hard requirement to work on user experience improvements of the Redash tool itself. Instead this GSoC project is targeted at web developers who enjoy improving tools that have a great impact on how Firefox is developed. Please also see the [https://docs.telemetry.mozilla.org/ Firefox data documentation] for a more detailed [https://docs.telemetry.mozilla.org/tools/stmo.html introduction into Mozilla's Redash instance]. A goal of this GSoC project is to upstream all improvements made to the upstream Redash project when possible.<br />
|-<br />
| Automatically detect web compatibility issues ([https://github.com/marco-c/autowebcompat autowebcompat])<br />
| Build a tool to automatically detect web compatibility issues using machine learning tools.<br />
| Python (knowing Keras or having experience with machine learning is a plus)<br />
| [mailto:marco@mozilla.com Marco (:marco)] (Mozilla)<br />
| [mailto:marco@mozilla.com Marco (:marco)] (Mozilla)<br />
| The project involves collecting data (screenshots of websites in different browsers), labeling it, training a neural network to automatically detect when a website is rendered differently in different browsers.<br />
|-<br />
| Autogeneration of style structs in Servo's style system<br />
| Prototype how to run [https://github.com/rlhunt/cbindgen cbindgen] on Servo's style system [https://github.com/servo/servo/tree/master/components/style/values values module] to auto-generate style struct definitions for C++ and Rust, which would enable the removal of 5k+ [https://github.com/servo/servo/blob/master/components/style/properties/gecko.mako.rs lines] of unsafe and slow Rust <-> C++ conversion code. This can be done on a per-struct basis, so prototyping the general mechanism with a simple one like [https://github.com/servo/servo/blob/master/components/style/properties/longhand/color.mako.rs color] would be the first step, migrating more and more as time goes on.<br />
| Primarily Rust (though no need to be really advanced), probably a bit Python for build system integration, and basic C++ since it needs to get generated.<br />
| [https://github.com/emilio emilio]<br />
| [https://github.com/emilio emilio]<br />
| [https://github.com/servo/servo/wiki/Autogeneration-of-style-structs Full project description]<br />
|-<br />
| Timely Security Analytics<br />
| InfoSec uses the Mozilla Defense Platform, MozDef to aggregate logs and alert on time series. This project seeks to create a structure for extract, transform, load operations (ETL) to process these time series events using MapReduce (Apache Spark). The project will have three milestones:<br />
* Demonstrated ability to load data from Mozilla SCL3, MDC1, MDC2 to GCP or AWS and process in apache spark. Student shall have choice of languages: scala, python, etc.<br />
* Demonstrated ability to reason about the MozDef standard data format using map reduce. Produce anomalies for known incident patterns in AWS CloudTrail logs. ( Example: An IAM user or service account used a service they have never used or logged in from a new GeoLocation / IP address tuple. )<br />
* Demonstrated abilility to reason about non structured data (non-MozDef standard format) and correlate with MozDef structured events.<br />
For more information see: [https://docs.google.com/document/d/1pzVbFw5TM8gG01Jabi4aOxAhiMvHGUHZTG-hV4dtAQs/edit?usp=sharing Google Doc]<br />
| Languages or skills needed: Python, Scala, Javascript, ETL, AWS/GCP, Data Science Fundamentals, Apache Spark, Pig, or other big data.<br />
| [https://mozillians.org/en-US/u/akrug/ Andrew Krug :akrug (mozilla)<br />
| [https://mozillians.org/en-US/u/akrug/ Andrew Krug :akrug (mozilla)<br />
[https://mozillians.org/en-US/u/michalpurzynski/] Michal Purzynski :michal` (mozilla)<br />
| This project represents a unique opportunity to work on one of the only fully open source SIEM projects in the community -- MozDef. The MozDef platform is running in production at Mozilla and ingests 14,000,000 events per year. The current alert system uses time series rules to generate meaningful alerts for the infosec team and Mozilla's Operations Center. We're hoping that with the help of qualified student(s) this rich data set can also generate alerts using basic machine learning techniques. This is a unique opportunity to work on aggregation of disparate sources of data: NSM, CloudTrail, etc. into spark. Real-time alerts over large quantities of data. <br />
|-<br />
| C++ Static Analysis<br />
| Add new checkers specific to our base code.<br />
| Strong C++ experience and clang infrastructure.<br />
| [mailto:s@mozilla.com Sylvestre]<br />
| [mailto:andi@mozilla.com Andi]<br />
| In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns). <br />
|-<br />
| Download Spam Protection for Firefox<br />
| In Firefox, we want websites to go through a permission prompt if they want to initiate several downloads in an automated (not user-initiated) fashion. The project will involve implementing a permission prompt, implementing the logic of when it should be triggered, and collecting data (via telemetry, etc) on how users interact with the prompt in order to verify how the feature behaves.<br />
| Good JavaScript experience and a basic understand of how the web works.<br />
| [mailto:jhofmann@mozilla.com Johann (:johannh)]<br />
| [mailto:nhnt11+gsoc@gmail.com Nihanth (:nhnt11)]<br />
| A more detailed description can be found [https://bugzilla.mozilla.org/show_bug.cgi?id=1306334#c2 here]<br />
|-<br />
| TLS scanning in Rust for Mozilla TLS Observatory <br />
| [https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to rewrite the [https://github.com/mozilla/cipherscan ciphersuite scanner] in Rust to support scanning TLS 1.3, but also maintain the ability to scan old version of TLS, all the way back to SSL2.0. This will likely require interacting with older versions of OpenSSL.<br />
<br />
Performance is also a strong requirement for this project. TLS Observatory scans thousands of sites every hour, and saving precious seconds on each scans make a big difference to the infrastructure.<br />
<br />
Finally, because this is a replacement of Cipherscan, this project requires maintaining backward compatibility with integrations that rely on specific flags, or json outputs.<br />
<br />
The final goal is to ship this project into the production infrastructure of TLS Observatory.<br />
| Programming skills in C and Rust. Strong understanding of TLS and micro-services architectures.<br />
| [mailto:jvehent@mozilla.com Julien Vehent]<br />
| [mailto:jvehent@mozilla.com Julien Vehent]<br />
| Cipherscan is the measuring tool that Mozilla uses to keep on eye on TLS configurations across the Internet. It is instrumental to the TLS guidelines work we do over at [https://wiki.mozilla.org/Security/Server_Side_TLS Server Side TLS]. This project will directly contribute to tooling that operators rely on worldwide to keep their websites secure.<br />
<br />
This is a fairly complex project that requires programming, networking and cryptography skills. Candidates will require skills in all three areas to succeed (GSoC is not long enough to learn them on the go).<br />
|-<br />
| Multi-tab select in Firefox<br />
| This project will focus on adding multiple-select functionality to the tab strip in Firefox. The selected student will teach Firefox to know about multiple tabs being selected, while also adding functionality to close, and move to new window.<br />
| JavaScript, CSS, and web experience<br />
| [mailto:jaws@mozilla.com Jared Wein (:jaws)]<br />
| [mailto:jaws@mozilla.com Jared Wein (:jaws)]<br />
| You can learn more by [https://bugzilla.mozilla.org/show_bug.cgi?id=566510 reading the bug on Bugzilla], but note that the comments there stem back multiple years as people have been wanting this feature for a long time. While working on this you will become very familiar with the [https://developer.mozilla.org/en-US/docs/Tools/Browser_Toolbox Browser Toolbox], our own "Inspector" tool for the browser user-interface, as well as writing automated tests for the new feature you're adding. Most of our tab-selection code lives within [https://searchfox.org/mozilla-central/source/browser/base/content/tabbrowser.xml our tabbrowser.xml] file, which you can browse around if you're interested to see how some operations work.<br />
|-<br />
| A/B testing framework for Android<br />
| In Firefox for Android we are using an A/B testing framework based on "Switchboard". We want to re-write this framework in Kotlin as a standalone-library that can be used in our other Android apps.<br />
| Android, Java, Kotlin<br />
| [mailto:skaspari@mozilla.com Sebastian Kaspari (:sebastian)]<br />
| [mailto:skaspari@mozilla.com Sebastian Kaspari (:sebastian)]<br />
| See more detailed project description:<br />
<br />
https://github.com/pocmo/fretboard/wiki/Google-Summer-of-Code-2018<br />
<br />
List of issues/tasks:<br />
<br />
https://github.com/pocmo/fretboard/issues<br />
|-<br />
| Native Android XML support in Mozilla localization infrastrucure<br />
| Mozilla develops software for various platforms, one of which is Android. Localizations of Android apps are typically stored in the XML dialect, which is not supported by [https://pontoon.mozilla.org/ Pontoon], Mozilla's localization tool. That means our localization process is unnecessarily complicated, because we need to convert between Android XML files and files supported by our localization tools. Your task will be to solve this problem by writing translation quality checks for [https://pypi.python.org/pypi/compare-locales compare-locales] and adding support for the XML dialect to Pontoon.<br />
| Python<br />
| [https://mozillians.org/en-US/u/mathjazz/ Matjaž Horvat] (Mozilla)<br />
| [https://mozillians.org/en-US/u/Pike/ Axel Hecht] (Mozilla)<br />
|-<br />
<br />
| Improve Enigmail User Experience<br />
| [https://enigmail.net Enigmail] is an add-on for Thunderbird that adds OpenPGP-based email encryption and authentication. This is a Thunderbird project that focuses on improving the user interface for Enigmail, such that it fits better into the general Thunderbird UI and makes user experience smoother. The goal of this project is to lower the entry-barrier for non-experts to use encryption by simplifying the existing core workflows and tasks. The main areas to work on are initial setup, sending and reading mails.<br />
| JavaScript, HTML/XUL and CSS<br />
| [mailto:patrick@enigmail.net Patrick Brunschwig (:pbrunschwig)], [mailto:kewisch@thunderbird.net Philipp Kewisch (:Fallen)]<br />
| [mailto:patrick@enigmail.net Patrick Brunschwig (:pbrunschwig)]<br />
| List your top 3 pain points in the Enigmail UI, and what you would do to fix them.<br />
|-<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode18&diff=1189665Community:SummerOfCode182018-03-03T03:39:38Z<p>Wlach: metricsgraphics project outline</p>
<hr />
<div>This is Mozilla's list of green-lit project proposals for the 2018 Google Summer of Code. <br />
<br />
<b>Are you a student looking to apply to GSoC with Mozilla?</b> You're in the right place. This page lists all the confirmed [[SummerOfCode|Google Summer of Code]] projects. New suggestions can be made on [[Community:SummerOfCode18:Brainstorming|the Brainstorming page]]. Do not edit this page yourself; contact Mike Hoye or Florian for edits.<br />
<br />
If you're interested in participating in Mozilla's GSoC program, you can choose from the list below, '''but you do not have to'''. You can submit a proposal for your own idea. You should look at the [[Community:SummerOfCode18:Brainstorming|guidelines]], though, and discuss your ideas or application in the #introduction channel on IRC.Mozilla.org. This is important, as GSoC projects '''must have''' a supporting member of the Mozilla community to evaluate and mentor them, named in the application.<br />
<br />
===Application Advice===<br />
<br />
You should do the following:<br />
<br />
* Talk to the mentor. Contact details are on this page; if all you have is a nickname, reach out to them on IRC.<br />
* Read the [https://flossmanuals.net/GSoCStudentGuide/ GSoC Student Guide] and follow its advice.<br />
* Read [http://blog.gerv.net/2006/05/how_not_to_apply_for_summer_of/ How Not To Apply For Summer Of Code] and avoid doing the things listed there.<br />
* Read our examples of good applications: [[SummerOfCode/SampleApplications/1|1]], [[SummerOfCode/SampleApplications/2|2]], [[SummerOfCode/SampleApplications/3|3]].<br />
* Apply on [https://summerofcode.withgoogle.com/ the GSoC site] (note that we have an [[SummerOfCode/ApplicationTemplate|application template]]).<br />
* It is entirely acceptable to apply for 2 or 3 projects, if more than one catches your eye; if the applications are high quality, that can improve your chances. Applying to more than that will seem like spam.<br />
<br />
Questions about individual projects are best addressed to the potential mentor of that project. These should be listed in the table below. If you want to contact a mentor and contact details are not here, ask people in the #introduction channel on IRC: irc://irc.mozilla.org/#introduction. If you have questions of any other sort, send mail to [mailto:mhoye@mozilla.com Mike Hoye]. He will try and respond promptly and direct your questions to the right person. <br />
<br />
===Project List===<br />
<br />
* The following projects have been tentatively greenlit, pending milestone clarifications.<br />
<br />
== 2018 Project List ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Additional Comments<br />
|-<br />
| ReSpec <br />
| [https://github.com/w3c/respec/ ReSpec] is a JS-based tool used to write W3C Specifications (Web Standards) that is widely used by the Web Standards Community. With 5+ years of development, it's heavily depended upon by the W3C community at large (of which Mozilla is an active participant). ReSpec's code is in need of some modernization, optimizations, and bug fixes - and we could use your help! In this project, you would have the opportunity to make ReSpec's UI more accessible, making it leaner and faster using distributed processing with Web Workers, and/or adding new features to make the lives of W3C spec Editor's better. <br />
| Languages or skills needed: JavaScript, HTML, CSS. <br />
| [https://mozillians.org/en-US/u/mcaceres/ Marcos Caceres]<br />
| [https://mozillians.org/en-US/u/mcaceres/ Marcos Caceres]<br />
| ReSpec offers students the opportunity to work on a large code base that has extensive real world use and impact. The project offers students an extensive range of problems to tackle, from UI design, to concurrent processing (using Web Workers to do distributed text processing), dealing with accessibility and internationalization, writing and learning about unit and integration tests, security, code review, etc. - as well as exposure to the W3C and the web standards community, this project also aims at teaching students about how web standards are put together. To determine if this is a project you would like to be part of, see the [https://github.com/w3c/respec/issues/ list of issues] you could work on. It's a great opportunity to learn about all aspects of open source software development, but with the freedom to take on small to large challenges over the Summer (depending on your skill level and level of confidence). About the mentor: Marcos Caceres is a Staff Engineer at Mozilla who has been working on Web Standards for over a decade. Marcos is the lead maintainer of ReSpec. Marcos has extensive experience mentoring developers and has previously successfully mentor a GSO student. <br />
|-<br />
| D3D11 backend for gfx-rs HAL<br />
| [https://github.com/gfx-rs/gfx gfx-rs] is a graphics abstraction library written in Rust and currently used for prototyping and investigation of [https://www.khronos.org/blog/khronos-announces-the-vulkan-portability-initiative Vulkan Portability] and [https://www.w3.org/community/gpu/ WebGPU] by Mozilla. The Hardware Abstraction Layer (HAL) of gfx-rs currently supports Vulkan, D3D12, Metal, and OpenGL. We want it to be powering [https://github.com/servo/webrender/ WebRender] for Firefox Quantum and Servo, and since WebRender currently runs on D3D11 (through [https://chromium.googlesource.com/angle/angle/+/master/README.md Angle]), we need to provide a native D3D11 backend. The implementation can be based off the existing D3D11 code in [https://github.com/gfx-rs/gfx/tree/pre-ll/src/backend/dx11 gfx-rs pre-HAL], and the main challenge is porting the code, optimizing, and ensuring it can successfully run the reference test suite.<br />
| Candidates should be familiar with Rust as well as low-level graphics development. Not having D3D11-specific experience is acceptable, given the willingness to learn it and a properly setup Windows development environment.<br />
| Dzmitry Malyshau (Mozilla)<br />
| Dzmitry Malyshau (Mozilla)<br />
| Proposal Timeline:<br />
* Before April 20: setup the Windows environment for DirectX development; run the examples and reference tests with existing Vulkan/DX12/GL backends; get in contact with developers on [https://gitter.im/gfx-rs/gfx Gitter].<br />
* April 20 – May 23: get familiar with gfx-rs HAL architecture, attempt to fix small entry-level issues and/or write an application/library on top of it.<br />
* May 23 – May 30: setup the backend skeleton with bits of actual logic, such as device initialization, and have it building on CI.<br />
* June 1 – July 5: implement the DX11 logic necessary to run the `quad` example and pass the basic graphics reference tests.<br />
* July 6 - July 31: implement support for compute and tessellation; benchmark the new backend against DX12; pass the full reference test suite.<br />
* Aug 1 - Aug 16: determine the list of capabilities that need to be exposed, merge it with GL backend legacy capabilities, expose to the reference tests.<br />
|-<br />
| Servo: Prototype ways of splitting up the script crate<br />
| [https://github.com/servo/servo/ Servo] is a web rendering engine written in Rust. One specific module contains a huge amount of code that is expensive to compile all at once. This project is intended to explore ways of splitting it into separate modules to achieve better build performance without breaking the complex set of interdependencies that exist in the code.<br />
| Candidates should be familiar with Rust, in particular using it in projects made of multiple crates and relying on features like traits and associated types.<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://github.com/servo/servo/wiki/Prototype-ways-of-splitting-the-script-crate Full project description page]<br />
|-<br />
| web-platform-tests: Improve test manifest workflow and performance<br />
| [https://github.com/w3c/web-platform-tests/ web-platform-tests] is a project for writing a cross-browser test suite, which is used across all major browser engines. The list of tests is stored in a JSON manifest file which is slow to generate and which we currently check in to the Firefox repo. These things create a poor developer experience. The goal of the project is to move to a system where the manifest is downloaded on demand for Firefox developers, and then to explore ways to speed up manifest generation such as employing parallelism or moving performance hotspots to Rust. <br />
| Candidates should be confident programming in Python.<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
| [https://github.com/jgraham/web-platform-tests/wiki/Improve-Test-Manifest-Workflow-and-Performance Full project description page]<br />
|-<br />
| WasmFiddle<br />
| [https://wasdk.github.io/WasmFiddle/ WasmFiddle] is a web application that facilitates learning of the WebAssembly and its usage in the browser. The goal is to create a learning tool that will allow user to quickly start a project that uses WebAssembly, learn based on examples, explore existing popular toolchains (emscripten, rust, etc), etc. The project will be similar to what jsfiddle, jsbin, or codepen proposes, but with additional elements specific to WebAssembly only, e.g. a code generation from C/C++ languages or how the code will be compiled by the browser and run in native machine code.<br />
| Languages or skills needed: JavaScript, HTML, CSS, WebAssembly. Suggested additional technologies: React, Rust, C++.<br />
| [mailto:ydelendik@mozilla.com Yury Delendik] :yury (Mozilla)<br />
| [mailto:ydelendik@mozilla.com Yury Delendik] :yury (Mozilla)<br />
| The project involves extending the functionality of the existing WasmFiddle/WasmExplorer projects to:<br />
* control, inspect and understand every part of the WebAssembly compilation pipeline;<br />
* don’t hide details or add magic build steps;<br />
* users can create, modify, fork and share fiddles, to reduce friction, no accounts;<br />
* influence JavaScript developers through well-thought out examples, best practices and templates;<br />
* add support for many languages: C/C++/Rust/etc;<br />
* create environments where users can play and have fun with: WebGL, Physics, Games, Audio Synthesizers.<br />
See [https://github.com/wasdk/WebAssemblyStudio/ WebAssemblyStudio repository] for new version of the WasmFiddle. <br />
|-<br />
| Pontoon's path to first contribution<br />
| Pontoon is Mozilla's localization tool, used by hundreds of contributors to translate Firefox, mozilla.org, and many other Mozilla products. It is critical to shipping Firefox to as many users as possible, all around the world. This year, we would like to improve the experience of first-time users of Pontoon. Your task would consist of defining a "path to first contribution", and making it as smooth as possible. That mainly involves adding a tutorial to walk users through the translation interface and reworking the landing page, all of that using recent front-end technologies in a fast-evolving application.<br />
| JavaScript, HTML, CSS, Python, UX. Django and React are pluses<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
|-<br />
| AOM - accessibility API<br />
| [https://github.com/WICG/aom/blob/gh-pages/explainer.md AOM] is JavaScript API designed to help the authors to make the web pages accessible, i.e. make them usable by the assisstive technologies such as screen readers, screen magnifiers and others. AOM specification is inspired by [https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA ARIA] standard, an HTML 5 extension, used to add semantics to the web pages. If you are familiar with ARIA, then you can think of AOM as JavaScript ARIA. The AOM proposal (phase 1) was already implemented in Chrome under a flag, and you can help do the same in Firefox.<br />
| Languages or skills needed: C++. <br />
| Alexander Surkov (Mozilla) (surkov.alexander@gmail.com)<br />
| Alexander Surkov (Mozilla) (surkov.alexander@gmail.com)<br />
| This project is the opportunity to hack on the heart of Mozilla Firefox, the Gecko engine. It involves working with a large code base and solving a wide variety of problems, from the solution architecture to finding efficient ways to implement it. If you want to learn about Gecko and accessibility, that's the right place. AOM has the potential of becoming a new standard in making the web page accessible next few years. You can help to happen it sooner.<br />
|-<br />
| metricsgraphics - zooming and brushing<br />
| [https://github.com/mozilla/metricsgraphics metricsgraphics] is JavaScript library designed to help with the presentation of interactive visualization. It is used extensively both inside and outside Mozilla. This project would be about adding a set of zooming and brushing features to facilitate the exploration of large and/or noisy datasets, such as in the [https://wiki.mozilla.org/EngineeringProductivity/Projects/Perfherder Perfherder] or [https://github.com/mozilla/missioncontrol/ Mission Control] projects. <br />
| Languages or skills needed: JavaScript, HTML/SVG<br />
| William Lachance (Mozilla)<br />
| William Lachance (Mozilla)<br />
| Detailed project outline: https://docs.google.com/document/d/1_KIOJtemqlCBktDdfdjDuS4XhICeaKO3QhNRwbnnf-g/edit?usp=sharing<br />
|-<br />
| Iodide Notebook<br />
| The [https://github.com/mozilla/iodide Iodide Notebook] is a pure client-side browser-based IDE and publication tool for scientific computing and computational inquiry using web technologies (HTML, CSS, Javascript, WebAssembly). It provides a Jupyter-style notebook interface, granting scientists full access to the power of the DOM and browser APIs within a familiar iterative workflow, and with zero overhead -- absolutely no software installation and configuration is required, so people can seamlessly learn from and extend the work of others.<br />
<br />
Students will contribute to: extending the functionality of the notebook; making the notebook code more robust and performant; helping to shape the user experience; and creating example notebooks.<br />
| JS; HTML; CSS; React/Redux; familiarity with at least one scientific computing language preferred (ex: MATLAB, Mathematica, Julia, Numpy/Scipy, R, etc). Science/math background is a major plus.<br />
| Brendan Colloran (bcolloran@mozilla.com)<br />
| Brendan Colloran (bcolloran@mozilla.com)<br />
| In addition to helping to build the notebook, actively dogfooding it will be an essential part of our work. This means that students with a scientific and/or applied math background will be encouraged to build example notebooks that demonstrate visualizations, simulations, data analyses, etc., on topics of their choosing. In addition to mentoring students on software development, we will provide mentorship in scientific computing and data science.<br />
|<br />
|-<br />
| Improved Pulse Inspector Backend<br />
| [https://github.com/taskcluster/taskcluster-rfcs/issues/104 More details here]. The idea is to create a backend service that will allow browser-side applications like [https://tools.taskcluster.net/pulse-inspector/ pulse inspector] to listen to arbitrary [http://pulseguardian.mozilla.org/ Pulse] messages.<br />
| JavaScript (server-side), AMQP, WebSockets<br />
| Dustin Mitchell (Mozilla)<br />
| Jonas Jensen (Mozilla) / Eli (Perelman Mozilla)<br />
| This project can largely be a generic read-only websocket proxy for RabbitMQ, refer to our [https://github.com/taskcluster/taskcluster-rfcs/issues/104 issue for more details].<br />
|-<br />
| Redash - Improve user experience of [https://sql.telemetry.mozilla.org STMO]<br />
| Mozilla runs an [https://sql.telemetry.mozilla.org/ instance] of the Open Source project [https://github.com/getredash/redash Redash], a data analysis and dashboarding tool that has been customized and configured for use with a number of the Firefox organization's data sets. It provides low barrier access to the vast amount of Firefox Telemetry data (and others) to make product decisions for Firefox and others projects.<br />
| Languages or skills needed: Python, JavaScript, HTML<br />
| [https://mozillians.org/en-US/u/jezdez/ Jannis Leidel (jezdez)] (Mozilla)<br />
| [https://mozillians.org/en-US/u/jezdez/ Jannis Leidel (jezdez)] (Mozilla)<br />
| Knowing SQL or having experience with data analysis is not a hard requirement to work on user experience improvements of the Redash tool itself. Instead this GSoC project is targeted at web developers who enjoy improving tools that have a great impact on how Firefox is developed. Please also see the [https://docs.telemetry.mozilla.org/ Firefox data documentation] for a more detailed [https://docs.telemetry.mozilla.org/tools/stmo.html introduction into Mozilla's Redash instance]. A goal of this GSoC project is to upstream all improvements made to the upstream Redash project when possible.<br />
|-<br />
| Automatically detect web compatibility issues ([https://github.com/marco-c/autowebcompat autowebcompat])<br />
| Build a tool to automatically detect web compatibility issues using machine learning tools.<br />
| Python (knowing Keras or having experience with machine learning is a plus)<br />
| [mailto:marco@mozilla.com Marco (:marco)] (Mozilla)<br />
| [mailto:marco@mozilla.com Marco (:marco)] (Mozilla)<br />
| The project involves collecting data (screenshots of websites in different browsers), labeling it, training a neural network to automatically detect when a website is rendered differently in different browsers.<br />
|-<br />
| Autogeneration of style structs in Servo's style system<br />
| Prototype how to run [https://github.com/rlhunt/cbindgen cbindgen] on Servo's style system [https://github.com/servo/servo/tree/master/components/style/values values module] to auto-generate style struct definitions for C++ and Rust, which would enable the removal of 5k+ [https://github.com/servo/servo/blob/master/components/style/properties/gecko.mako.rs lines] of unsafe and slow Rust <-> C++ conversion code. This can be done on a per-struct basis, so prototyping the general mechanism with a simple one like [https://github.com/servo/servo/blob/master/components/style/properties/longhand/color.mako.rs color] would be the first step, migrating more and more as time goes on.<br />
| Primarily Rust (though no need to be really advanced), probably a bit Python for build system integration, and basic C++ since it needs to get generated.<br />
| [https://github.com/emilio emilio]<br />
| [https://github.com/emilio emilio]<br />
| [https://github.com/servo/servo/wiki/Autogeneration-of-style-structs Full project description]<br />
|-<br />
| Timely Security Analytics<br />
| InfoSec uses the Mozilla Defense Platform, MozDef to aggregate logs and alert on time series. This project seeks to create a structure for extract, transform, load operations (ETL) to process these time series events using MapReduce (Apache Spark). The project will have three milestones:<br />
* Demonstrated ability to load data from Mozilla SCL3, MDC1, MDC2 to GCP or AWS and process in apache spark. Student shall have choice of languages: scala, python, etc.<br />
* Demonstrated ability to reason about the MozDef standard data format using map reduce. Produce anomalies for known incident patterns in AWS CloudTrail logs. ( Example: An IAM user or service account used a service they have never used or logged in from a new GeoLocation / IP address tuple. )<br />
* Demonstrated abilility to reason about non structured data (non-MozDef standard format) and correlate with MozDef structured events.<br />
For more information see: [https://docs.google.com/document/d/1pzVbFw5TM8gG01Jabi4aOxAhiMvHGUHZTG-hV4dtAQs/edit?usp=sharing Google Doc]<br />
| Languages or skills needed: Python, Scala, Javascript, ETL, AWS/GCP, Data Science Fundamentals, Apache Spark, Pig, or other big data.<br />
| [https://mozillians.org/en-US/u/akrug/ Andrew Krug :akrug (mozilla)<br />
| [https://mozillians.org/en-US/u/akrug/ Andrew Krug :akrug (mozilla)<br />
[https://mozillians.org/en-US/u/michalpurzynski/] Michal Purzynski :michal` (mozilla)<br />
| This project represents a unique opportunity to work on one of the only fully open source SIEM projects in the community -- MozDef. The MozDef platform is running in production at Mozilla and ingests 14,000,000 events per year. The current alert system uses time series rules to generate meaningful alerts for the infosec team and Mozilla's Operations Center. We're hoping that with the help of qualified student(s) this rich data set can also generate alerts using basic machine learning techniques. This is a unique opportunity to work on aggregation of disparate sources of data: NSM, CloudTrail, etc. into spark. Real-time alerts over large quantities of data. <br />
|-<br />
| C++ Static Analysis<br />
| Add new checkers specific to our base code.<br />
| Strong C++ experience and clang infrastructure.<br />
| [mailto:s@mozilla.com Sylvestre]<br />
| [mailto:andi@mozilla.com Andi]<br />
| In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns). <br />
|-<br />
| Download Spam Protection for Firefox<br />
| In Firefox, we want websites to go through a permission prompt if they want to initiate several downloads in an automated (not user-initiated) fashion. The project will involve implementing a permission prompt, implementing the logic of when it should be triggered, and collecting data (via telemetry, etc) on how users interact with the prompt in order to verify how the feature behaves.<br />
| Good JavaScript experience and a basic understand of how the web works.<br />
| [mailto:jhofmann@mozilla.com Johann (:johannh)]<br />
| [mailto:nhnt11+gsoc@gmail.com Nihanth (:nhnt11)]<br />
| A more detailed description can be found [https://bugzilla.mozilla.org/show_bug.cgi?id=1306334#c2 here]<br />
|-<br />
| TLS scanning in Rust for Mozilla TLS Observatory <br />
| [https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to rewrite the [https://github.com/mozilla/cipherscan ciphersuite scanner] in Rust to support scanning TLS 1.3, but also maintain the ability to scan old version of TLS, all the way back to SSL2.0. This will likely require interacting with older versions of OpenSSL.<br />
<br />
Performance is also a strong requirement for this project. TLS Observatory scans thousands of sites every hour, and saving precious seconds on each scans make a big difference to the infrastructure.<br />
<br />
Finally, because this is a replacement of Cipherscan, this project requires maintaining backward compatibility with integrations that rely on specific flags, or json outputs.<br />
<br />
The final goal is to ship this project into the production infrastructure of TLS Observatory.<br />
| Programming skills in C and Rust. Strong understanding of TLS and micro-services architectures.<br />
| [mailto:jvehent@mozilla.com Julien Vehent]<br />
| [mailto:jvehent@mozilla.com Julien Vehent]<br />
| Cipherscan is the measuring tool that Mozilla uses to keep on eye on TLS configurations across the Internet. It is instrumental to the TLS guidelines work we do over at [https://wiki.mozilla.org/Security/Server_Side_TLS Server Side TLS]. This project will directly contribute to tooling that operators rely on worldwide to keep their websites secure.<br />
<br />
This is a fairly complex project that requires programming, networking and cryptography skills. Candidates will require skills in all three areas to succeed (GSoC is not long enough to learn them on the go).<br />
|-<br />
| Multi-tab select in Firefox<br />
| This project will focus on adding multiple-select functionality to the tab strip in Firefox. The selected student will teach Firefox to know about multiple tabs being selected, while also adding functionality to close, and move to new window.<br />
| JavaScript, CSS, and web experience<br />
| [mailto:jaws@mozilla.com Jared Wein (:jaws)]<br />
| [mailto:jaws@mozilla.com Jared Wein (:jaws)]<br />
| You can learn more by [https://bugzilla.mozilla.org/show_bug.cgi?id=566510 reading the bug on Bugzilla], but note that the comments there stem back multiple years as people have been wanting this feature for a long time. While working on this you will become very familiar with the [https://developer.mozilla.org/en-US/docs/Tools/Browser_Toolbox Browser Toolbox], our own "Inspector" tool for the browser user-interface, as well as writing automated tests for the new feature you're adding. Most of our tab-selection code lives within [https://searchfox.org/mozilla-central/source/browser/base/content/tabbrowser.xml our tabbrowser.xml] file, which you can browse around if you're interested to see how some operations work.<br />
|-<br />
| A/B testing framework for Android<br />
| In Firefox for Android we are using an A/B testing framework based on "Switchboard". We want to re-write this framework in Kotlin as a standalone-library that can be used in our other Android apps.<br />
| Android, Java, Kotlin<br />
| [mailto:skaspari@mozilla.com Sebastian Kaspari (:sebastian)]<br />
| [mailto:skaspari@mozilla.com Sebastian Kaspari (:sebastian)]<br />
| See more detailed project description:<br />
<br />
https://github.com/pocmo/fretboard/wiki/Google-Summer-of-Code-2018<br />
<br />
List of issues/tasks:<br />
<br />
https://github.com/pocmo/fretboard/issues<br />
|-<br />
| Native Android XML support in Mozilla localization infrastrucure<br />
| Mozilla develops software for various platforms, one of which is Android. Localizations of Android apps are typically stored in the XML dialect, which is not supported by [https://pontoon.mozilla.org/ Pontoon], Mozilla's localization tool. That means our localization process is unnecessarily complicated, because we need to convert between Android XML files and files supported by our localization tools. Your task will be to solve this problem by writing translation quality checks for [https://pypi.python.org/pypi/compare-locales compare-locales] and adding support for the XML dialect to Pontoon.<br />
| Python<br />
| [https://mozillians.org/en-US/u/mathjazz/ Matjaž Horvat] (Mozilla)<br />
| [https://mozillians.org/en-US/u/Pike/ Axel Hecht] (Mozilla)<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode18:Brainstorming&diff=1187078Community:SummerOfCode18:Brainstorming2018-01-18T22:22:30Z<p>Wlach: metricsgraphics stub</p>
<hr />
<div>Mozilla community members - submit proposals here for 2018 Google Summer of Code projects with Mozilla. (If this page looks empty, it's because accepted ideas have already been transferred to the [[Community:SummerOfCode18|official list]].) Our application for summer of 2018 goes into Google on '''January 23rd''', so get those project ideas in quickly.<br />
<br />
<b>Are you a student looking to apply to SoC with Mozilla?</b> Your first stop should be the [[Community:SummerOfCode18|official list of ideas]]. This page is full of weird ideas, some of which are still on here for a reason - it could be that they are not properly defined, the wrong size, or don't have a mentor. That makes them less likely to get accepted. You <i>can</i>, of course, also submit your own ideas - you don't have to put an idea on this page and get it 'made official' in order to send in a proposal for it.<br />
<br />
==How To Write A Good Project Proposal==<br />
<br />
Before adding an proposal to this list, please consider the following:<br />
<br />
* '''Be specific'''. It's hard to understand the impact of, or the size of, vague proposals.<br />
* '''Consider size'''. The student has eight weeks to design, code, test and document the proposal. It needs to fill, but not overfill, that time.<br />
* '''Do your research'''. Support the idea with well-researched links.<br />
* '''Don't morph other people's ideas'''. If you have a related idea, place it next to the existing one, or add a comment. <br />
* '''Insert only your own name into the Mentor column''', and then only if you are willing to take on the responsibility. If you think the SoC admins won't know who you are, leave contact details.<br />
* '''Check back regularly'''. The administrators may have questions about your idea that you will need to answer.<br />
* '''Know when to give up'''. If you've added the same idea for the last three years and it hasn't made it to the official page, perhaps you can predict what will happen this time.<br />
<br />
==Suggestion List==<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
Here is the template for adding project proposals. Feel free to add new proposal to existing general categories, or add new categories as appropriate. <br />
<br />
== General category or project name ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Title of your proposal<br />
| A few sentences describing the project.<br />
| Languages or skills needed<br />
| Project submitter's info<br />
| Project mentor's info<br />
| Anything else people might need to know about this submission.<br />
|-<br />
| ReSpec <br />
| [https://github.com/w3c/respec/ ReSpec] is a JS-based tool used to write W3C Specifications (Web Standards) that is widely used by the Web Standards Community. With 5+ years of development, it's heavily depended upon by the W3C community at large (of which Mozilla is an active participant). ReSpec's code is in need of some modernization, optimizations, and bug fixes - and we could use your help! In this project, you would have the opportunity to make ReSpec's UI more accessible, making it leaner and faster using distributed processing with Web Workers, and/or adding new features to make the lives of W3C spec Editor's better. <br />
| Languages or skills needed: JavaScript, HTML, CSS. <br />
| Marcos Caceres (Mozilla)<br />
| Marcos Caceres (Mozilla)<br />
| ReSpec offers students the opportunity to work on a large code base that has extensive real world use and impact. The project offers students an extensive range of problems to tackle, from UI design, to concurrent processing (using Web Workers to do distributed text processing), dealing with accessibility and internationalization, writing and learning about unit and integration tests, security, code review, etc. - as well as exposure to the W3C and the web standards community, this project also aims at teaching students about how web standards are put together. To determine if this is a project you would like to be part of, see the [https://github.com/w3c/respec/issues/ list of issues] you could work on. It's a great opportunity to learn about all aspects of open source software development, but with the freedom to take on small to large challenges over the Summer (depending on your skill level and level of confidence). About the mentor: Marcos Caceres is a Staff Engineer at Mozilla who has been working on Web Standards for over a decade. Marcos is the lead maintainer of ReSpec. Marcos has extensive experience mentoring developers and has previously successfully mentor a GSO student. <br />
|-<br />
| D3D11 backend for gfx-rs HAL<br />
| [https://github.com/gfx-rs/gfx gfx-rs] is a graphics abstraction library written in Rust and currently used for prototyping and investigation of [https://www.khronos.org/blog/khronos-announces-the-vulkan-portability-initiative Vulkan Portability] and [https://www.w3.org/community/gpu/ WebGPU] by Mozilla. The Hardware Abstraction Layer (HAL) of gfx-rs currently supports Vulkan, D3D12, Metal, and OpenGL. We want it to be powering [https://github.com/servo/webrender/ WebRender] for Firefox Quantum and Servo, and since WebRender currently runs on D3D11 (through [https://chromium.googlesource.com/angle/angle/+/master/README.md Angle]), we need to provide a native D3D11 backend. The implementation can be based off the existing D3D11 backend of [https://github.com/gfx-rs/gfx/tree/pre-ll/src/backend/dx11 gfx-rs pre-HAL], and the main challenge is porting the code, optimizing, and testing it.<br />
| Candidates should be familiar with Rust as well as low-level graphics development. Not having D3D11-specific experience is acceptable, given the willingness to learn it and a properly setup Windows development environment.<br />
| Dzmitry Malyshau (Mozilla)<br />
| Dzmitry Malyshau (Mozilla)<br />
| gfx-rs is one of the foundational projects in Rust gamedev/graphics, with rich history and wide range of users. We have an active community that encourages learning and curiosity in how graphics works down to the hardware. This is not a Mozilla project, but it may become a key component in Firefox Quantum for the ability to run natively on different platforms, while utilizing the fine-grained control of the graphics/compute workloads exposed by the low-level APIs.<br />
|-<br />
| Servo: Prototype ways of splitting up the script crate<br />
| [https://github.com/servo/servo/ Servo] is a web rendering engine written in Rust. One specific module contains a huge amount of code that is expensive to compile all at once. This project is intended to explore ways of splitting it into separate modules to achieve better build performance without breaking the complex set of interdependencies that exist in the code.<br />
| Candidates should be familiar with Rust, in particular using it in projects made of multiple crates and relying on features like traits and associated types.<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://mozillians.org/en-US/u/jdm/ Josh Matthews] (Mozilla)<br />
| [https://github.com/servo/servo/wiki/Prototype-ways-of-splitting-the-script-crate Full project description page]<br />
|-<br />
| web-platform-tests: Improve test manifest workflow and performance<br />
| [https://github.com/w3c/web-platform-tests/ web-platform-tests] is a project for writing a cross-browser test suite, which is used across all major browser engines. The list of tests is stored in a JSON manifest file which is slow to generate and which we currently check in to the Firefox repo. These things create a poor developer experience. The goal of the project is to move to a system where the manifest is downloaded on demand for Firefox developers, and then to explore ways to speed up manifest generation such as employing parallelism or moving performance hotspots to Rust. <br />
| Candidates should be confident programming in Python.<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
| [https://github.com/jgraham James Graham] (Mozilla)<br />
|-<br />
| WasmFiddle<br />
| [https://wasdk.github.io/WasmFiddle/ WasmFiddle] is a web application that facilitates learning of the WebAssembly and its usage in the browser. The goal is to create a learning tool that will allow user to quickly start a project that uses WebAssembly, learn based on examples, explore existing popular toolchains (emscripten, rust, etc), etc. The project will be similar to what jsfiddle, jsbin, or codepen proposes, but with additional elements specific to WebAssembly only, e.g. a code generation from C/C++ languages or how the code will be compiled by the browser and run in native machine code.<br />
| Languages or skills needed: JavaScript, HTML, CSS, WebAssembly. Suggested additional technologies: React, Rust, C++.<br />
| Yury Delendik (Mozilla)<br />
| Yury Delendik (Mozilla)<br />
| The project involves extending the functionality of the existing WasmFiddle/WasmExplorer projects to:<br />
* control, inspect and understand every part of the WebAssembly compilation pipeline;<br />
* don’t hide details or add magic build steps;<br />
* users can create, modify, fork and share fiddles, to reduce friction, no accounts;<br />
* influence JavaScript developers through well-thought out examples, best practices and templates;<br />
* add support for many languages: C/C++/Rust/etc;<br />
* create environments where users can play and have fun with: WebGL, Physics, Games, Audio Synthesizers<br />
|-<br />
| Pontoon's path to first contribution<br />
| Pontoon is Mozilla's localization tool, used by hundreds of contributors to translate Firefox, mozilla.org, and many other Mozilla products. It is critical to shipping Firefox to as many users as possible, all around the world. This year, we would like to improve the experience of first-time users of Pontoon. Your task would consist of defining a "path to first contribution", and making it as smooth as possible. That mainly involves adding a tutorial to walk users through the translation interface and reworking the landing page, all of that using recent front-end technologies in a fast-evolving application.<br />
| JavaScript, HTML, CSS, Python, UX. Django and React are pluses<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
| [https://mozillians.org/en-US/u/adrian/ Adrian Gaudebert] (Mozilla)<br />
|<br />
|-<br />
| AOM - accessibility API<br />
| [https://github.com/WICG/aom/blob/gh-pages/explainer.md AOM] is JavaScript API designed to help the authors to make the web pages accessible, i.e. make them usable by the assisstive technologies such as screen readers, screen magnifiers and others. AOM specification is inspired by [https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA ARIA] standard, an HTML 5 extension, used to add semantics to the web pages. If you are familiar with ARIA, then you can think of AOM as JavaScript ARIA. The AOM proposal (phase 1) was already implemented in Chrome under a flag, and you can help do the same in Firefox.<br />
| Languages or skills needed: C++. <br />
| Alexander Surkov (Mozilla)<br />
| Alexander Surkov (Mozilla)<br />
| This project is the opportunity to hack on the heart of Mozilla Firefox, the Gecko engine. It involves working with a large code base and solving a wide variety of problems, from the solution architecture to finding efficient ways to implement it. If you want to learn about Gecko and accessibility, that's the right place. AOM has the potential of becoming a new standard in making the web page accessible next few years. You can help to happen it sooner.<br />
|-<br />
| metricsgraphics - zooming and brushing<br />
| [https://github.com/mozilla/metricsgraphics metricsgraphics] is JavaScript library designed to help with the presentation of interactive visualization. It is used extensively both inside and outside Mozilla. This project would be about adding a set of zooming and brushing features to facilitate the exploration of large and/or noisy datasets, such as in the [https://wiki.mozilla.org/EngineeringProductivity/Projects/Perfherder Perfherder] or [https://github.com/mozilla/missioncontrol/ Mission Control] projects. Detailed project description to come.<br />
| Languages or skills needed: JavaScript, HTML/SVG<br />
| William Lachance (Mozilla)<br />
| William Lachance (Mozilla)<br />
| <br />
|-<br />
| Another proposal, etc.<br />
|<br />
|<br />
|<br />
|<br />
|<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry/Custom_analysis_with_spark&diff=1171115Telemetry/Custom analysis with spark2017-05-15T15:05:44Z<p>Wlach: a few notes about zeppelin notebooks</p>
<hr />
<div><br />
== Introduction ==<br />
Spark is a data processing engine designed to be fast and easy to use. We have setup Jupyter workbooks that use Spark to analyze our Telemetry data. Jupyter workbooks can be easily shared and updated among colleagues to enable richer analysis than SQL alone.<br />
<br />
The Spark clusters can be spun up on analysis.telemetry.mozilla.org, which is abbreviated as atmo. The Spark Python API is called pyspark.<br />
<br />
== Setting Up a Spark Cluster On atmo ==<br />
<br />
# Go to analysis.telemetry.mozilla.org<br />
# Click “Launch an ad-hoc Spark cluster”.<br />
# Enter some details:<br />
## The “Cluster Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Set the number of workers for the cluster. Please keep in mind to use resources sparingly; use a single worker to write and debug your job.<br />
## Upload your SSH public key.<br />
# Click “Submit”.<br />
# A cluster will be launched on AWS preconfigured with Spark, IPython and some handy data analysis libraries like pandas and matplotlib.<br />
<br />
Once the cluster is ready, you can tunnel IPython through SSH by following the instructions on the dashboard, and running the ssh shell command. For example:<br />
<br />
ssh -i ~/.ssh/id_rsa -L 8888:localhost:8888 hadoop@ec2-54-70-129-221.us-west-2.compute.amazonaws.com<br />
<br />
Finally, you can launch IPython in Firefox by visiting http://localhost:8888.<br />
<br />
== The Python Jupyter Notebook ==<br />
When you access http://localhost:8888, two example Jupyter notebooks are available to peruse. To create a new Jupyter notebook, select new -> Python 2.<br />
<br />
Starting out, we recommend looking through the [https://github.com/mozilla/mozilla-reports/blob/master/tutorials/telemetry_hello_world.kp/orig_src/Telemetry%20Hello%20World.ipynb Telemetry Hello World] notebook. It gives a nice overview of Jupyter and analyzing telemetry data using pyspark and the RDD API.<br />
<br />
=== Using Jupyter ===<br />
Jupyter Notebooks contain a series of cells. Each cell contains code or markdown. To switch between the two, use the dropdown at the top. To run a cell, use shift-enter; this either compiles the markdown or runs the code. To create new cell, select Insert -> Insert Cell Below.<br />
<br />
A cell can output text or plots. To output plots inlined with the cell, run the following command, usually below your import statements:<br />
%pylab inline<br />
<br />
The notebook is setup to work with Spark. See the "Using Spark" section for more information.<br />
<br />
=== Schedule a periodic job ===<br />
<br />
Scheduled Spark jobs allow a Jupyter notebook to be updated consistently, making a nice and easy-to-use dashboard. <br />
<br />
To schedule a Spark job:<br />
<br />
# Visit the analysis provisioning dashboard at telemetry-dash.mozilla.org and sign in using Persona with an @mozilla.com email address.<br />
# Click “Schedule a Spark Job”.<br />
# Enter some details:<br />
## The “Job Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Upload your IPython notebook containing the analysis.<br />
## Set the number of workers of the cluster in the “Cluster Size” field.<br />
## Set a schedule frequency using the remaining fields.<br />
<br />
Now, the notebook will be updated automatically and the results can be easily shared. Furthermore, all files stored in the notebook's local working directory at the end of the job will be automatically uploaded to S3, which comes in handy for simple ETL workloads for example.<br />
<br />
For reference, see [https://robertovitillo.com/2015/03/13/simple-dashboards-with-scheduled-spark-jobs-and-plotly Simple Dashboard with Scheduled Spark Jobs and Plotly].<br />
<br />
=== Sharing a Notebook ===<br />
<br />
Jupyter notebooks can be shared in a few different ways.<br />
<br />
==== Sharing a Static Notebook ====<br />
<br />
An easy way to share is using a gist on github.<br />
<br />
# Download file as .ipynb<br />
# Upload to a gist on [http://gist.github.com gist.github.com]<br />
# Enter the gist URL at [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
# Share with your colleagues!<br />
<br />
==== Sharing a Scheduled Notebook ====<br />
<br />
Setup your scheduled notebook. After it's run, do the following:<br />
# Go to the 'Schedule a Spark job' tab in atmo<br />
# Get the URL for the notebook (under 'Currently Scheduled Jobs')<br />
# Paste that URL into [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
<br />
== Zeppelin Notebooks ==<br />
<br />
We also have *experimental* support for [http://zeppelin.apache.org/ Apache Zeppelin] notebooks. The notebook server for that is running on port 8890, so you can connect to it just by tunneling the port (instead of port 8888 for Jupyter). For example:<br />
<br />
ssh -i ~/.ssh/id_rsa -L 8890:localhost:8890 hadoop@ec2-54-70-129-221.us-west-2.compute.amazonaws.com <br />
<br />
== Using Spark ==<br />
Spark is a general-purpose cluster computing system - it allows users to run general execution graphs. APIs are available in Python, Scala, and Java. The Jupyter notebook utilizes the Python API. In a nutshell, it provides a way to run functional code (e.g. map, reduce, etc.) on large, distributed data.<br />
<br />
Check out [https://robertovitillo.com/2015/06/30/spark-best-practices/ Spark Best Practices] for tips on using Spark to it's full capabilities.<br />
<br />
=== SparkContext (sc) ===<br />
Access to the Spark API is provided through SparkContext. In the Jupyter notebook, this is the `sc` object. For example, to create a distributed RDD of monotonically increasing numbers 1-1000:<br />
numbers = range(1000)<br />
#no need to initialize sc in the Jupyter notebook<br />
numsRdd = sc.parallelize(numbers)<br />
nums.take(10) #no guaranteed order<br />
<br />
=== Spark RDD ===<br />
The Resilient Distributed Dataset (RDD) is Spark's basic data structure. The operations that are performed on these structures are distributed to the cluster. Only certain actions (such as collect() or take(N)) pull an RDD in locally.<br />
<br />
RDD's are nice because there is no imposed schema - whatever they contain, they distribute around the cluster. Additionally, RDD's can be cached in memory, which can greatly improve performance of some algorithms that need access to data over and over again. <br />
<br />
Additionally, RDD operations are all part of a directed, acyclic graph. This gives increased redundancy, since Spark is always able to recreate an RDD from the base data (by rerunning the graph), but also provides lazy evaluation. No computation is performed while an RDD is just being transformed (a la map), but when an action is taken (e.g. reduce, take) the entire computation graph is evaluated. Continuing from our previous example, the following gives some of the peaks of a sin wave:<br />
<br />
import numpy as np<br />
#no computation is performed on the following line!<br />
sin_values = numsRdd.map(lambda x : np.float(x) / 10).map(lambda x : (x, np.sin(x)))<br />
#now the entire computation graph is evaluated<br />
sin_values.takeOrdered(5, lambda x : -x[1])<br />
<br />
For jumping into working with Spark RDD's, we recommend reading the [https://spark.apache.org/docs/latest/programming-guide.html Spark Programming Guide].<br />
<br />
=== Spark SQL and Spark Dataframes/Datasets ===<br />
Spark also supports traditional SQL, along with special data structures that require schemas. The Spark SQL API can be accessed with the `spark` object. For example:<br />
<br />
longitudinal = spark.sql('SELECT * FROM longitudinal')<br />
<br />
creates a DataFrame that contains all the longitudinal data. A Spark DataFrame is essentially a distributed table, a la Pandas or R Dataframes. Under the covers they are an RDD of Row objects, and thus the entirety of the RDD API is available for DataFrames, as well as a DataFrame specific API. For example, a sql-like way to get the count of a specific OS:<br />
<br />
longitudinal.select("os").where("os = 'Darwin'").count()<br />
<br />
To Transform the DataFrame object to an RDD, simply do:<br />
<br />
longitudinal_rdd = longitudinal.rdd<br />
<br />
In general, however, the DataFrames are performance optimized, so it's worth the effort to learn the DataFrame API.<br />
<br />
For more overview, see the [https://spark.apache.org/docs/latest/sql-programming-guide.html SQL Programming Guide]. See also the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Longitudinal%20Dataset%20Tutorial.ipynb Longitudinal Tutorial], one of the available example notebooks when you start a cluster.<br />
<br />
=== Available Data Sources for SparkSQL ===<br />
For information about available queryable data sources (e.g. Longitudinal dataset), see [https://wiki.mozilla.org/Telemetry/Available_Telemetry_Datasets_and_their_Applications Telemetry Datasets Documentation].<br />
<br />
These datasets are optimized for fast access, and will far out-perform analysis on the raw telemetry ping data.<br />
<br />
===Accessing the Spark UI===<br />
Go to localhost:8888/spark after ssh-ing into the spark cluster to see the Spark UI. It has information about job statuses and task completion, and may help you debug your job.<br />
<br />
== The MozTelemetry Library ==<br />
We have provided a library that gives easy access to the raw telemetry ping data. For example usage, see the [http://reports.telemetry.mozilla.org/post/tutorials/telemetry_hello_world.kp Telemetry Hello World] example notebook. Detailed documentation for the library can be found at the [http://python-moztelemetry.readthedocs.io Python MozTelemetry Documentation].<br />
<br />
=== Using the Raw Ping Data ===<br />
First off, import the moztelemetry library using the following:<br />
<br />
from moztelemetry import get_pings<br />
<br />
If you need any other functions, just comma separate them with get_pings.<br />
<br />
The ping data is an RDD of JSON elements. For example, using the following:<br />
<br />
pings = get_pings(sc, app="Firefox", channel="nightly", build_id=("20160901000000", "20160901999999"), fraction=0.01)<br />
<br />
returns an RDD of 1/100th of Firefox Nightly JSON pings for all builds from September 1 2016. Now, because it's JSON, pings are easy to access. For example, to get the count of each OS type:<br />
<br />
os_names = pings.map(lambda x : (x['environment']['system']['os']['name'], 1))<br />
os_counts = os_names.reduceByKey(lambda x, y : x+y)<br />
os_counts.collect()<br />
<br />
Alternatively, moztelemetry provides the `get_pings_properties` function, which will gather the data for you:<br />
<br />
subset = get_pings_properties(pings, ["environment/system/os/name"])<br />
subset.map(lambda x : (x["environment/system/os/name"], 1)).reduceByKey(lambda x, y : x+y).collect()<br />
<br />
== FAQ ==<br />
Please add more FAQ as questions are answered by you or for you.<br />
<br />
=== How can I load parquet datasets in a Jupyter notebook? ===<br />
Use spark.read.parquet, e.g.:<br />
<br />
dataset = spark.read.parquet("s3://the_bucket/the_prefix/the_version")<br />
<br />
=== I got a REMOTE HOST IDENTIFICATION HAS CHANGED! error ===<br />
AWS recycles hostnames, so removing the offending key from $HOME/.ssh/known_hosts will remove the warning. You can find the line to remove by finding the line in the output that says<br />
Offending key in /path/to/hosts/known_hosts:2<br />
Where 2 is the line number of the key that can be deleted. Just remove that line, save the file, and try again.<br />
<br />
=== Why is my notebook hanging? ===<br />
There are a couple common causes for this:<br />
<br />
1. Currently, our Spark notebooks can only run a single Python kernel at a time. If you open multiple notebooks on the same cluster and try to run both, the second notebook will hang. Be sure to close notebooks using "Close and Halt" under the "File" dropdown.<br />
<br />
2. The connection from PySpark to the Spark driver might be lost. Unfortunately the best way to recover from this for the moment seems to be spinning up a new cluster.<br />
<br />
3. Canceling execution of a notebook cell doesn't cancel any spark jobs that might be running in the background. If your spark commands seem to be hanging, try running `sc.cancelAllJobs()`.<br />
<br />
=== How can I keep running after closing the notebook? ===<br />
For long-running computation, it might be nice to close the notebook (and the ssh session) and look at the results later. Unfortunately, '''all cell output will be lost when a notebook is closed''' (for the running cell). To alleviate this, there are a few options:<br />
<br />
1. Have everything output to a variable. These values should still be available when you reconnect.<br />
<br />
2. Put %%capture at the beginning of the cell to store all output. [https://ipython.org/ipython-doc/3/interactive/magics.html#cellmagic-capture See the documentation].<br />
<br />
=== How do I load an external library into the cluster? ===<br />
Assuming you've got a url for the repo, you can create an egg for it this way:<br />
<br />
!git clone <repo url> && cd <repo-name> && python setup.py bdist_egg<br />
sc.addPyFile('<repo-name>/dist/my-egg-file.egg')<br />
<br />
Alternately, you could just create that egg locally, upload it to a web server, then download and install it:<br />
<br />
import requests<br />
r = requests.get('<url-to-my-egg-file>')<br />
with open('mylibrary.egg', 'wb') as f:<br />
f.write(r.content)<br />
sc.addPyFile('mylibrary.egg')<br />
<br />
You will want to do this '''before''' you load the library. If the library is already loaded, restart the kernel in the ipython notebook.</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry/Custom_analysis_with_spark&diff=1171074Telemetry/Custom analysis with spark2017-05-12T18:36:26Z<p>Wlach: /* The MozTelemetry Library */ update other link to hello world notebook</p>
<hr />
<div><br />
== Introduction ==<br />
Spark is a data processing engine designed to be fast and easy to use. We have setup Jupyter workbooks that use Spark to analyze our Telemetry data. Jupyter workbooks can be easily shared and updated among colleagues to enable richer analysis than SQL alone.<br />
<br />
The Spark clusters can be spun up on analysis.telemetry.mozilla.org, which is abbreviated as atmo. The Spark Python API is called pyspark.<br />
<br />
== Setting Up a Spark Cluster On atmo ==<br />
<br />
# Go to analysis.telemetry.mozilla.org<br />
# Click “Launch an ad-hoc Spark cluster”.<br />
# Enter some details:<br />
## The “Cluster Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Set the number of workers for the cluster. Please keep in mind to use resources sparingly; use a single worker to write and debug your job.<br />
## Upload your SSH public key.<br />
# Click “Submit”.<br />
# A cluster will be launched on AWS preconfigured with Spark, IPython and some handy data analysis libraries like pandas and matplotlib.<br />
<br />
Once the cluster is ready, you can tunnel IPython through SSH by following the instructions on the dashboard, and running the ssh shell command. For example:<br />
<br />
ssh -i ~/.ssh/id_rsa -L 8888:localhost:8888 hadoop@ec2-54-70-129-221.us-west-2.compute.amazonaws.com<br />
<br />
Finally, you can launch IPython in Firefox by visiting http://localhost:8888.<br />
<br />
== The Python Jupyter Notebook ==<br />
When you access http://localhost:8888, two example Jupyter notebooks are available to peruse. To create a new Jupyter notebook, select new -> Python 2.<br />
<br />
Starting out, we recommend looking through the [https://github.com/mozilla/mozilla-reports/blob/master/tutorials/telemetry_hello_world.kp/orig_src/Telemetry%20Hello%20World.ipynb Telemetry Hello World] notebook. It gives a nice overview of Jupyter and analyzing telemetry data using pyspark and the RDD API.<br />
<br />
=== Using Jupyter ===<br />
Jupyter Notebooks contain a series of cells. Each cell contains code or markdown. To switch between the two, use the dropdown at the top. To run a cell, use shift-enter; this either compiles the markdown or runs the code. To create new cell, select Insert -> Insert Cell Below.<br />
<br />
A cell can output text or plots. To output plots inlined with the cell, run the following command, usually below your import statements:<br />
%pylab inline<br />
<br />
The notebook is setup to work with Spark. See the "Using Spark" section for more information.<br />
<br />
=== Schedule a periodic job ===<br />
<br />
Scheduled Spark jobs allow a Jupyter notebook to be updated consistently, making a nice and easy-to-use dashboard. <br />
<br />
To schedule a Spark job:<br />
<br />
# Visit the analysis provisioning dashboard at telemetry-dash.mozilla.org and sign in using Persona with an @mozilla.com email address.<br />
# Click “Schedule a Spark Job”.<br />
# Enter some details:<br />
## The “Job Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Upload your IPython notebook containing the analysis.<br />
## Set the number of workers of the cluster in the “Cluster Size” field.<br />
## Set a schedule frequency using the remaining fields.<br />
<br />
Now, the notebook will be updated automatically and the results can be easily shared. Furthermore, all files stored in the notebook's local working directory at the end of the job will be automatically uploaded to S3, which comes in handy for simple ETL workloads for example.<br />
<br />
For reference, see [https://robertovitillo.com/2015/03/13/simple-dashboards-with-scheduled-spark-jobs-and-plotly Simple Dashboard with Scheduled Spark Jobs and Plotly].<br />
<br />
=== Sharing a Notebook ===<br />
<br />
Jupyter notebooks can be shared in a few different ways.<br />
<br />
==== Sharing a Static Notebook ====<br />
<br />
An easy way to share is using a gist on github.<br />
<br />
# Download file as .ipynb<br />
# Upload to a gist on [http://gist.github.com gist.github.com]<br />
# Enter the gist URL at [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
# Share with your colleagues!<br />
<br />
==== Sharing a Scheduled Notebook ====<br />
<br />
Setup your scheduled notebook. After it's run, do the following:<br />
# Go to the 'Schedule a Spark job' tab in atmo<br />
# Get the URL for the notebook (under 'Currently Scheduled Jobs')<br />
# Paste that URL into [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
<br />
== Using Spark ==<br />
Spark is a general-purpose cluster computing system - it allows users to run general execution graphs. APIs are available in Python, Scala, and Java. The Jupyter notebook utilizes the Python API. In a nutshell, it provides a way to run functional code (e.g. map, reduce, etc.) on large, distributed data.<br />
<br />
Check out [https://robertovitillo.com/2015/06/30/spark-best-practices/ Spark Best Practices] for tips on using Spark to it's full capabilities.<br />
<br />
=== SparkContext (sc) ===<br />
Access to the Spark API is provided through SparkContext. In the Jupyter notebook, this is the `sc` object. For example, to create a distributed RDD of monotonically increasing numbers 1-1000:<br />
numbers = range(1000)<br />
#no need to initialize sc in the Jupyter notebook<br />
numsRdd = sc.parallelize(numbers)<br />
nums.take(10) #no guaranteed order<br />
<br />
=== Spark RDD ===<br />
The Resilient Distributed Dataset (RDD) is Spark's basic data structure. The operations that are performed on these structures are distributed to the cluster. Only certain actions (such as collect() or take(N)) pull an RDD in locally.<br />
<br />
RDD's are nice because there is no imposed schema - whatever they contain, they distribute around the cluster. Additionally, RDD's can be cached in memory, which can greatly improve performance of some algorithms that need access to data over and over again. <br />
<br />
Additionally, RDD operations are all part of a directed, acyclic graph. This gives increased redundancy, since Spark is always able to recreate an RDD from the base data (by rerunning the graph), but also provides lazy evaluation. No computation is performed while an RDD is just being transformed (a la map), but when an action is taken (e.g. reduce, take) the entire computation graph is evaluated. Continuing from our previous example, the following gives some of the peaks of a sin wave:<br />
<br />
import numpy as np<br />
#no computation is performed on the following line!<br />
sin_values = numsRdd.map(lambda x : np.float(x) / 10).map(lambda x : (x, np.sin(x)))<br />
#now the entire computation graph is evaluated<br />
sin_values.takeOrdered(5, lambda x : -x[1])<br />
<br />
For jumping into working with Spark RDD's, we recommend reading the [https://spark.apache.org/docs/latest/programming-guide.html Spark Programming Guide].<br />
<br />
=== Spark SQL and Spark Dataframes/Datasets ===<br />
Spark also supports traditional SQL, along with special data structures that require schemas. The Spark SQL API can be accessed with the `spark` object. For example:<br />
<br />
longitudinal = spark.sql('SELECT * FROM longitudinal')<br />
<br />
creates a DataFrame that contains all the longitudinal data. A Spark DataFrame is essentially a distributed table, a la Pandas or R Dataframes. Under the covers they are an RDD of Row objects, and thus the entirety of the RDD API is available for DataFrames, as well as a DataFrame specific API. For example, a sql-like way to get the count of a specific OS:<br />
<br />
longitudinal.select("os").where("os = 'Darwin'").count()<br />
<br />
To Transform the DataFrame object to an RDD, simply do:<br />
<br />
longitudinal_rdd = longitudinal.rdd<br />
<br />
In general, however, the DataFrames are performance optimized, so it's worth the effort to learn the DataFrame API.<br />
<br />
For more overview, see the [https://spark.apache.org/docs/latest/sql-programming-guide.html SQL Programming Guide]. See also the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Longitudinal%20Dataset%20Tutorial.ipynb Longitudinal Tutorial], one of the available example notebooks when you start a cluster.<br />
<br />
=== Available Data Sources for SparkSQL ===<br />
For information about available queryable data sources (e.g. Longitudinal dataset), see [https://wiki.mozilla.org/Telemetry/Available_Telemetry_Datasets_and_their_Applications Telemetry Datasets Documentation].<br />
<br />
These datasets are optimized for fast access, and will far out-perform analysis on the raw telemetry ping data.<br />
<br />
===Accessing the Spark UI===<br />
Go to localhost:8888/spark after ssh-ing into the spark cluster to see the Spark UI. It has information about job statuses and task completion, and may help you debug your job.<br />
<br />
== The MozTelemetry Library ==<br />
We have provided a library that gives easy access to the raw telemetry ping data. For example usage, see the [http://reports.telemetry.mozilla.org/post/tutorials/telemetry_hello_world.kp Telemetry Hello World] example notebook. Detailed documentation for the library can be found at the [http://python-moztelemetry.readthedocs.io Python MozTelemetry Documentation].<br />
<br />
=== Using the Raw Ping Data ===<br />
First off, import the moztelemetry library using the following:<br />
<br />
from moztelemetry import get_pings<br />
<br />
If you need any other functions, just comma separate them with get_pings.<br />
<br />
The ping data is an RDD of JSON elements. For example, using the following:<br />
<br />
pings = get_pings(sc, app="Firefox", channel="nightly", build_id=("20160901000000", "20160901999999"), fraction=0.01)<br />
<br />
returns an RDD of 1/100th of Firefox Nightly JSON pings for all builds from September 1 2016. Now, because it's JSON, pings are easy to access. For example, to get the count of each OS type:<br />
<br />
os_names = pings.map(lambda x : (x['environment']['system']['os']['name'], 1))<br />
os_counts = os_names.reduceByKey(lambda x, y : x+y)<br />
os_counts.collect()<br />
<br />
Alternatively, moztelemetry provides the `get_pings_properties` function, which will gather the data for you:<br />
<br />
subset = get_pings_properties(pings, ["environment/system/os/name"])<br />
subset.map(lambda x : (x["environment/system/os/name"], 1)).reduceByKey(lambda x, y : x+y).collect()<br />
<br />
== FAQ ==<br />
Please add more FAQ as questions are answered by you or for you.<br />
<br />
=== How can I load parquet datasets in a Jupyter notebook? ===<br />
Use spark.read.parquet, e.g.:<br />
<br />
dataset = spark.read.parquet("s3://the_bucket/the_prefix/the_version")<br />
<br />
=== I got a REMOTE HOST IDENTIFICATION HAS CHANGED! error ===<br />
AWS recycles hostnames, so removing the offending key from $HOME/.ssh/known_hosts will remove the warning. You can find the line to remove by finding the line in the output that says<br />
Offending key in /path/to/hosts/known_hosts:2<br />
Where 2 is the line number of the key that can be deleted. Just remove that line, save the file, and try again.<br />
<br />
=== Why is my notebook hanging? ===<br />
There are a couple common causes for this:<br />
<br />
1. Currently, our Spark notebooks can only run a single Python kernel at a time. If you open multiple notebooks on the same cluster and try to run both, the second notebook will hang. Be sure to close notebooks using "Close and Halt" under the "File" dropdown.<br />
<br />
2. The connection from PySpark to the Spark driver might be lost. Unfortunately the best way to recover from this for the moment seems to be spinning up a new cluster.<br />
<br />
3. Canceling execution of a notebook cell doesn't cancel any spark jobs that might be running in the background. If your spark commands seem to be hanging, try running `sc.cancelAllJobs()`.<br />
<br />
=== How can I keep running after closing the notebook? ===<br />
For long-running computation, it might be nice to close the notebook (and the ssh session) and look at the results later. Unfortunately, '''all cell output will be lost when a notebook is closed''' (for the running cell). To alleviate this, there are a few options:<br />
<br />
1. Have everything output to a variable. These values should still be available when you reconnect.<br />
<br />
2. Put %%capture at the beginning of the cell to store all output. [https://ipython.org/ipython-doc/3/interactive/magics.html#cellmagic-capture See the documentation].<br />
<br />
=== How do I load an external library into the cluster? ===<br />
Assuming you've got a url for the repo, you can create an egg for it this way:<br />
<br />
!git clone <repo url> && cd <repo-name> && python setup.py bdist_egg<br />
sc.addPyFile('<repo-name>/dist/my-egg-file.egg')<br />
<br />
Alternately, you could just create that egg locally, upload it to a web server, then download and install it:<br />
<br />
import requests<br />
r = requests.get('<url-to-my-egg-file>')<br />
with open('mylibrary.egg', 'wb') as f:<br />
f.write(r.content)<br />
sc.addPyFile('mylibrary.egg')<br />
<br />
You will want to do this '''before''' you load the library. If the library is already loaded, restart the kernel in the ipython notebook.</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry/Custom_analysis_with_spark&diff=1167013Telemetry/Custom analysis with spark2017-03-29T15:41:33Z<p>Wlach: /* How do I load an external library into the cluster? */ remove redundant steps</p>
<hr />
<div><br />
== Introduction ==<br />
Spark is a data processing engine designed to be fast and easy to use. We have setup Jupyter workbooks that use Spark to analyze our Telemetry data. Jupyter workbooks can be easily shared and updated among colleagues to enable richer analysis than SQL alone.<br />
<br />
The Spark clusters can be spun up on analysis.telemetry.mozilla.org, which is abbreviated as atmo. The Spark Python API is called pyspark.<br />
<br />
== Setting Up a Spark Cluster On atmo ==<br />
<br />
# Go to analysis.telemetry.mozilla.org<br />
# Click “Launch an ad-hoc Spark cluster”.<br />
# Enter some details:<br />
## The “Cluster Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Set the number of workers for the cluster. Please keep in mind to use resources sparingly; use a single worker to write and debug your job.<br />
## Upload your SSH public key.<br />
# Click “Submit”.<br />
# A cluster will be launched on AWS preconfigured with Spark, IPython and some handy data analysis libraries like pandas and matplotlib.<br />
<br />
Once the cluster is ready, you can tunnel IPython through SSH by following the instructions on the dashboard, and running the ssh shell command. For example:<br />
<br />
ssh -i ~/.ssh/id_rsa -L 8888:localhost:8888 hadoop@ec2-54-70-129-221.us-west-2.compute.amazonaws.com<br />
<br />
Finally, you can launch IPython in Firefox by visiting http://localhost:8888.<br />
<br />
== The Python Jupyter Notebook ==<br />
When you access http://localhost:8888, two example Jupyter notebooks are available to peruse. To create a new Jupyter notebook, select new -> Python 2.<br />
<br />
Starting out, we recommend looking through the [https://github.com/mozilla/mozilla-reports/blob/master/tutorials/telemetry_hello_world.kp/orig_src/Telemetry%20Hello%20World.ipynb Telemetry Hello World] notebook. It gives a nice overview of Jupyter and analyzing telemetry data using pyspark and the RDD API.<br />
<br />
=== Using Jupyter ===<br />
Jupyter Notebooks contain a series of cells. Each cell contains code or markdown. To switch between the two, use the dropdown at the top. To run a cell, use shift-enter; this either compiles the markdown or runs the code. To create new cell, select Insert -> Insert Cell Below.<br />
<br />
A cell can output text or plots. To output plots inlined with the cell, run the following command, usually below your import statements:<br />
%pylab inline<br />
<br />
The notebook is setup to work with Spark. See the "Using Spark" section for more information.<br />
<br />
=== Schedule a periodic job ===<br />
<br />
Scheduled Spark jobs allow a Jupyter notebook to be updated consistently, making a nice and easy-to-use dashboard. <br />
<br />
To schedule a Spark job:<br />
<br />
# Visit the analysis provisioning dashboard at telemetry-dash.mozilla.org and sign in using Persona with an @mozilla.com email address.<br />
# Click “Schedule a Spark Job”.<br />
# Enter some details:<br />
## The “Job Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Upload your IPython notebook containing the analysis.<br />
## Set the number of workers of the cluster in the “Cluster Size” field.<br />
## Set a schedule frequency using the remaining fields.<br />
<br />
Now, the notebook will be updated automatically and the results can be easily shared. Furthermore, all files stored in the notebook's local working directory at the end of the job will be automatically uploaded to S3, which comes in handy for simple ETL workloads for example.<br />
<br />
For reference, see [https://robertovitillo.com/2015/03/13/simple-dashboards-with-scheduled-spark-jobs-and-plotly Simple Dashboard with Scheduled Spark Jobs and Plotly].<br />
<br />
=== Sharing a Notebook ===<br />
<br />
Jupyter notebooks can be shared in a few different ways.<br />
<br />
==== Sharing a Static Notebook ====<br />
<br />
An easy way to share is using a gist on github.<br />
<br />
# Download file as .ipynb<br />
# Upload to a gist on [http://gist.github.com gist.github.com]<br />
# Enter the gist URL at [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
# Share with your colleagues!<br />
<br />
==== Sharing a Scheduled Notebook ====<br />
<br />
Setup your scheduled notebook. After it's run, do the following:<br />
# Go to the 'Schedule a Spark job' tab in atmo<br />
# Get the URL for the notebook (under 'Currently Scheduled Jobs')<br />
# Paste that URL into [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
<br />
== Using Spark ==<br />
Spark is a general-purpose cluster computing system - it allows users to run general execution graphs. APIs are available in Python, Scala, and Java. The Jupyter notebook utilizes the Python API. In a nutshell, it provides a way to run functional code (e.g. map, reduce, etc.) on large, distributed data.<br />
<br />
Check out [https://robertovitillo.com/2015/06/30/spark-best-practices/ Spark Best Practices] for tips on using Spark to it's full capabilities.<br />
<br />
=== SparkContext (sc) ===<br />
Access to the Spark API is provided through SparkContext. In the Jupyter notebook, this is the `sc` object. For example, to create a distributed RDD of monotonically increasing numbers 1-1000:<br />
numbers = range(1000)<br />
#no need to initialize sc in the Jupyter notebook<br />
numsRdd = sc.parallelize(numbers)<br />
nums.take(10) #no guaranteed order<br />
<br />
=== Spark RDD ===<br />
The Resilient Distributed Dataset (RDD) is Spark's basic data structure. The operations that are performed on these structures are distributed to the cluster. Only certain actions (such as collect() or take(N)) pull an RDD in locally.<br />
<br />
RDD's are nice because there is no imposed schema - whatever they contain, they distribute around the cluster. Additionally, RDD's can be cached in memory, which can greatly improve performance of some algorithms that need access to data over and over again. <br />
<br />
Additionally, RDD operations are all part of a directed, acyclic graph. This gives increased redundancy, since Spark is always able to recreate an RDD from the base data (by rerunning the graph), but also provides lazy evaluation. No computation is performed while an RDD is just being transformed (a la map), but when an action is taken (e.g. reduce, take) the entire computation graph is evaluated. Continuing from our previous example, the following gives some of the peaks of a sin wave:<br />
<br />
import numpy as np<br />
#no computation is performed on the following line!<br />
sin_values = numsRdd.map(lambda x : np.float(x) / 10).map(lambda x : (x, np.sin(x)))<br />
#now the entire computation graph is evaluated<br />
sin_values.takeOrdered(5, lambda x : -x[1])<br />
<br />
For jumping into working with Spark RDD's, we recommend reading the [https://spark.apache.org/docs/latest/programming-guide.html Spark Programming Guide].<br />
<br />
=== Spark SQL and Spark Dataframes/Datasets ===<br />
Spark also supports traditional SQL, along with special data structures that require schemas. The Spark SQL API can be accessed with `sqlContext`. For example:<br />
<br />
longitudinal = sqlContext.sql('SELECT * FROM longitudinal')<br />
<br />
creates a DataFrame that contains all the longitudinal data. A Spark DataFrame is essentially a distributed table, a la Pandas or R Dataframes. Under the covers they are an RDD of Row objects, and thus the entirety of the RDD API is available for DataFrames, as well as a DataFrame specific API. For example, a sql-like way to get the count of a specific OS:<br />
<br />
longitudinal.select("os").where("os = 'Darwin'").count()<br />
<br />
To Transform the DataFrame object to an RDD, simply do:<br />
<br />
longitudinal_rdd = longitudinal.rdd<br />
<br />
In general, however, the DataFrames are performance optimized, so it's worth the effort to learn the DataFrame API.<br />
<br />
For more overview, see the [https://spark.apache.org/docs/latest/sql-programming-guide.html SQL Programming Guide]. See also the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Longitudinal%20Dataset%20Tutorial.ipynb Longitudinal Tutorial], one of the available example notebooks when you start a cluster.<br />
<br />
=== Available Data Sources for SparkSQL ===<br />
For information about available queryable data sources (e.g. Longitudinal dataset), see [https://wiki.mozilla.org/Telemetry/Available_Telemetry_Datasets_and_their_Applications Telemetry Datasets Documentation].<br />
<br />
These datasets are optimized for fast access, and will far out-perform analysis on the raw telemetry ping data.<br />
<br />
===Accessing the Spark UI===<br />
Go to localhost:8888/spark after sshing into the spark cluster to see the Spark UI. It has information about job statuses and task completion, and may help you debug your job.<br />
<br />
== The MozTelemetry Library ==<br />
We have provided a library that gives easy access to the raw telemetry ping data. For example usage, see the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Telemetry%20Hello%20World.ipynb Telemetry Hello World] example notebook. Detailed documentation for the library can be found at the [http://python-moztelemetry.readthedocs.io Python MozTelemetry Documentation].<br />
<br />
=== Using the Raw Ping Data ===<br />
First off, import the moztelemetry library using the following:<br />
<br />
from moztelemetry import get_pings<br />
<br />
If you need any other functions, just comma separate them with get_pings.<br />
<br />
The ping data is an RDD of JSON elements. For example, using the following:<br />
<br />
pings = get_pings(sc, app="Firefox", channel="nightly", build_id=("20160901000000", "20160901999999"), fraction=0.01)<br />
<br />
returns an RDD of 1/100th of Firefox Nightly JSON pings for all builds from September 1 2016. Now, because it's JSON, pings are easy to access. For example, to get the count of each OS type:<br />
<br />
os_names = pings.map(lambda x : (x['environment']['system']['os']['name'], 1))<br />
os_counts = os_names.reduceByKey(lambda x, y : x+y)<br />
os_counts.collect()<br />
<br />
Alternatively, moztelemetry provides the `get_pings_properties` function, which will gather the data for you:<br />
<br />
subset = get_pings_properties(pings, ["environment/system/os/name"])<br />
subset.map(lambda x : (x["environment/system/os/name"], 1)).reduceByKey(lambda x, y : x+y).collect()<br />
<br />
== FAQ ==<br />
Please add more FAQ as questions are answered by you or for you.<br />
<br />
=== How can I load parquet datasets in a Jupyter notebook? ===<br />
Use sqlContext.read.load, e.g.:<br />
<br />
dataset = sqlContext.read.load("s3://the_bucket/the_prefix/the_version", "parquet")<br />
<br />
=== I got a REMOTE HOST IDENTIFICATION HAS CHANGED! error ===<br />
AWS recycles hostnames, so removing the offending key from $HOME/.ssh/known_hosts will remove the warning. You can find the line to remove by finding the line in the output that says<br />
Offending key in /path/to/hosts/known_hosts:2<br />
Where 2 is the line number of the key that can be deleted. Just remove that line, save the file, and try again.<br />
<br />
=== Why is my notebook hanging? ===<br />
There are a couple common causes for this:<br />
<br />
1. Currently, our Spark notebooks can only run a single Python kernel at a time. If you open multiple notebooks on the same cluster and try to run both, the second notebook will hang. Be sure to close notebooks using "Close and Halt" under the "File" dropdown.<br />
<br />
2. The connection from PySpark to the Spark driver might be lost. Unfortunately the best way to recover from this for the moment seems to be spinning up a new cluster.<br />
<br />
3. Canceling execution of a notebook cell doesn't cancel any spark jobs that might be running in the background. If your spark commands seem to be hanging, try running `sc.cancelAllJobs()`.<br />
<br />
=== How can I keep running after closing the notebook? ===<br />
For long-running computation, it might be nice to close the notebook (and the ssh session) and look at the results later. Unfortunately, '''all cell output will be lost when a notebook is closed''' (for the running cell). To alleviate this, there are a few options:<br />
<br />
1. Have everything output to a variable. These values should still be available when you reconnect.<br />
<br />
2. Put %%capture at the beginning of the cell to store all output. [https://ipython.org/ipython-doc/3/interactive/magics.html#cellmagic-capture See the documentation].<br />
<br />
=== How do I load an external library into the cluster? ===<br />
Assuming you've got a url for the repo, you can create an egg for it this way:<br />
<br />
!git clone <repo url> && cd <repo-name> && python setup.py bdist_egg<br />
sc.addPyFile('<repo-name>/dist/my-egg-file.egg')<br />
<br />
Alternately, you could just create that egg locally, upload it to a web server, then download and install it:<br />
<br />
import requests<br />
r = requests.get('<url-to-my-egg-file>')<br />
with open('mylibrary.egg', 'wb') as f:<br />
f.write(r.content)<br />
sc.addPyFile('mylibrary.egg')<br />
<br />
You will want to do this '''before''' you load the library. If the library is already loaded, restart the kernel in the ipython notebook.</div>Wlachhttps://wiki.mozilla.org/index.php?title=Telemetry/Custom_analysis_with_spark&diff=1167005Telemetry/Custom analysis with spark2017-03-29T15:08:17Z<p>Wlach: /* How do I load an external library into the cluster? */ Update external library loading instructions to include ipython context + alternate egg downloading method</p>
<hr />
<div><br />
== Introduction ==<br />
Spark is a data processing engine designed to be fast and easy to use. We have setup Jupyter workbooks that use Spark to analyze our Telemetry data. Jupyter workbooks can be easily shared and updated among colleagues to enable richer analysis than SQL alone.<br />
<br />
The Spark clusters can be spun up on analysis.telemetry.mozilla.org, which is abbreviated as atmo. The Spark Python API is called pyspark.<br />
<br />
== Setting Up a Spark Cluster On atmo ==<br />
<br />
# Go to analysis.telemetry.mozilla.org<br />
# Click “Launch an ad-hoc Spark cluster”.<br />
# Enter some details:<br />
## The “Cluster Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Set the number of workers for the cluster. Please keep in mind to use resources sparingly; use a single worker to write and debug your job.<br />
## Upload your SSH public key.<br />
# Click “Submit”.<br />
# A cluster will be launched on AWS preconfigured with Spark, IPython and some handy data analysis libraries like pandas and matplotlib.<br />
<br />
Once the cluster is ready, you can tunnel IPython through SSH by following the instructions on the dashboard, and running the ssh shell command. For example:<br />
<br />
ssh -i ~/.ssh/id_rsa -L 8888:localhost:8888 hadoop@ec2-54-70-129-221.us-west-2.compute.amazonaws.com<br />
<br />
Finally, you can launch IPython in Firefox by visiting http://localhost:8888.<br />
<br />
== The Python Jupyter Notebook ==<br />
When you access http://localhost:8888, two example Jupyter notebooks are available to peruse. To create a new Jupyter notebook, select new -> Python 2.<br />
<br />
Starting out, we recommend looking through the [https://github.com/mozilla/mozilla-reports/blob/master/tutorials/telemetry_hello_world.kp/orig_src/Telemetry%20Hello%20World.ipynb Telemetry Hello World] notebook. It gives a nice overview of Jupyter and analyzing telemetry data using pyspark and the RDD API.<br />
<br />
=== Using Jupyter ===<br />
Jupyter Notebooks contain a series of cells. Each cell contains code or markdown. To switch between the two, use the dropdown at the top. To run a cell, use shift-enter; this either compiles the markdown or runs the code. To create new cell, select Insert -> Insert Cell Below.<br />
<br />
A cell can output text or plots. To output plots inlined with the cell, run the following command, usually below your import statements:<br />
%pylab inline<br />
<br />
The notebook is setup to work with Spark. See the "Using Spark" section for more information.<br />
<br />
=== Schedule a periodic job ===<br />
<br />
Scheduled Spark jobs allow a Jupyter notebook to be updated consistently, making a nice and easy-to-use dashboard. <br />
<br />
To schedule a Spark job:<br />
<br />
# Visit the analysis provisioning dashboard at telemetry-dash.mozilla.org and sign in using Persona with an @mozilla.com email address.<br />
# Click “Schedule a Spark Job”.<br />
# Enter some details:<br />
## The “Job Name” field should be a short descriptive name, like “chromehangs analysis”.<br />
## Upload your IPython notebook containing the analysis.<br />
## Set the number of workers of the cluster in the “Cluster Size” field.<br />
## Set a schedule frequency using the remaining fields.<br />
<br />
Now, the notebook will be updated automatically and the results can be easily shared. Furthermore, all files stored in the notebook's local working directory at the end of the job will be automatically uploaded to S3, which comes in handy for simple ETL workloads for example.<br />
<br />
For reference, see [https://robertovitillo.com/2015/03/13/simple-dashboards-with-scheduled-spark-jobs-and-plotly Simple Dashboard with Scheduled Spark Jobs and Plotly].<br />
<br />
=== Sharing a Notebook ===<br />
<br />
Jupyter notebooks can be shared in a few different ways.<br />
<br />
==== Sharing a Static Notebook ====<br />
<br />
An easy way to share is using a gist on github.<br />
<br />
# Download file as .ipynb<br />
# Upload to a gist on [http://gist.github.com gist.github.com]<br />
# Enter the gist URL at [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
# Share with your colleagues!<br />
<br />
==== Sharing a Scheduled Notebook ====<br />
<br />
Setup your scheduled notebook. After it's run, do the following:<br />
# Go to the 'Schedule a Spark job' tab in atmo<br />
# Get the URL for the notebook (under 'Currently Scheduled Jobs')<br />
# Paste that URL into [http://nbviewer.jupyter.org/ Jupyter nbviewer]<br />
<br />
== Using Spark ==<br />
Spark is a general-purpose cluster computing system - it allows users to run general execution graphs. APIs are available in Python, Scala, and Java. The Jupyter notebook utilizes the Python API. In a nutshell, it provides a way to run functional code (e.g. map, reduce, etc.) on large, distributed data.<br />
<br />
Check out [https://robertovitillo.com/2015/06/30/spark-best-practices/ Spark Best Practices] for tips on using Spark to it's full capabilities.<br />
<br />
=== SparkContext (sc) ===<br />
Access to the Spark API is provided through SparkContext. In the Jupyter notebook, this is the `sc` object. For example, to create a distributed RDD of monotonically increasing numbers 1-1000:<br />
numbers = range(1000)<br />
#no need to initialize sc in the Jupyter notebook<br />
numsRdd = sc.parallelize(numbers)<br />
nums.take(10) #no guaranteed order<br />
<br />
=== Spark RDD ===<br />
The Resilient Distributed Dataset (RDD) is Spark's basic data structure. The operations that are performed on these structures are distributed to the cluster. Only certain actions (such as collect() or take(N)) pull an RDD in locally.<br />
<br />
RDD's are nice because there is no imposed schema - whatever they contain, they distribute around the cluster. Additionally, RDD's can be cached in memory, which can greatly improve performance of some algorithms that need access to data over and over again. <br />
<br />
Additionally, RDD operations are all part of a directed, acyclic graph. This gives increased redundancy, since Spark is always able to recreate an RDD from the base data (by rerunning the graph), but also provides lazy evaluation. No computation is performed while an RDD is just being transformed (a la map), but when an action is taken (e.g. reduce, take) the entire computation graph is evaluated. Continuing from our previous example, the following gives some of the peaks of a sin wave:<br />
<br />
import numpy as np<br />
#no computation is performed on the following line!<br />
sin_values = numsRdd.map(lambda x : np.float(x) / 10).map(lambda x : (x, np.sin(x)))<br />
#now the entire computation graph is evaluated<br />
sin_values.takeOrdered(5, lambda x : -x[1])<br />
<br />
For jumping into working with Spark RDD's, we recommend reading the [https://spark.apache.org/docs/latest/programming-guide.html Spark Programming Guide].<br />
<br />
=== Spark SQL and Spark Dataframes/Datasets ===<br />
Spark also supports traditional SQL, along with special data structures that require schemas. The Spark SQL API can be accessed with `sqlContext`. For example:<br />
<br />
longitudinal = sqlContext.sql('SELECT * FROM longitudinal')<br />
<br />
creates a DataFrame that contains all the longitudinal data. A Spark DataFrame is essentially a distributed table, a la Pandas or R Dataframes. Under the covers they are an RDD of Row objects, and thus the entirety of the RDD API is available for DataFrames, as well as a DataFrame specific API. For example, a sql-like way to get the count of a specific OS:<br />
<br />
longitudinal.select("os").where("os = 'Darwin'").count()<br />
<br />
To Transform the DataFrame object to an RDD, simply do:<br />
<br />
longitudinal_rdd = longitudinal.rdd<br />
<br />
In general, however, the DataFrames are performance optimized, so it's worth the effort to learn the DataFrame API.<br />
<br />
For more overview, see the [https://spark.apache.org/docs/latest/sql-programming-guide.html SQL Programming Guide]. See also the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Longitudinal%20Dataset%20Tutorial.ipynb Longitudinal Tutorial], one of the available example notebooks when you start a cluster.<br />
<br />
=== Available Data Sources for SparkSQL ===<br />
For information about available queryable data sources (e.g. Longitudinal dataset), see [https://wiki.mozilla.org/Telemetry/Available_Telemetry_Datasets_and_their_Applications Telemetry Datasets Documentation].<br />
<br />
These datasets are optimized for fast access, and will far out-perform analysis on the raw telemetry ping data.<br />
<br />
===Accessing the Spark UI===<br />
Go to localhost:8888/spark after sshing into the spark cluster to see the Spark UI. It has information about job statuses and task completion, and may help you debug your job.<br />
<br />
== The MozTelemetry Library ==<br />
We have provided a library that gives easy access to the raw telemetry ping data. For example usage, see the [https://github.com/mozilla/emr-bootstrap-spark/blob/master/examples/Telemetry%20Hello%20World.ipynb Telemetry Hello World] example notebook. Detailed documentation for the library can be found at the [http://python-moztelemetry.readthedocs.io Python MozTelemetry Documentation].<br />
<br />
=== Using the Raw Ping Data ===<br />
First off, import the moztelemetry library using the following:<br />
<br />
from moztelemetry import get_pings<br />
<br />
If you need any other functions, just comma separate them with get_pings.<br />
<br />
The ping data is an RDD of JSON elements. For example, using the following:<br />
<br />
pings = get_pings(sc, app="Firefox", channel="nightly", build_id=("20160901000000", "20160901999999"), fraction=0.01)<br />
<br />
returns an RDD of 1/100th of Firefox Nightly JSON pings for all builds from September 1 2016. Now, because it's JSON, pings are easy to access. For example, to get the count of each OS type:<br />
<br />
os_names = pings.map(lambda x : (x['environment']['system']['os']['name'], 1))<br />
os_counts = os_names.reduceByKey(lambda x, y : x+y)<br />
os_counts.collect()<br />
<br />
Alternatively, moztelemetry provides the `get_pings_properties` function, which will gather the data for you:<br />
<br />
subset = get_pings_properties(pings, ["environment/system/os/name"])<br />
subset.map(lambda x : (x["environment/system/os/name"], 1)).reduceByKey(lambda x, y : x+y).collect()<br />
<br />
== FAQ ==<br />
Please add more FAQ as questions are answered by you or for you.<br />
<br />
=== How can I load parquet datasets in a Jupyter notebook? ===<br />
Use sqlContext.read.load, e.g.:<br />
<br />
dataset = sqlContext.read.load("s3://the_bucket/the_prefix/the_version", "parquet")<br />
<br />
=== I got a REMOTE HOST IDENTIFICATION HAS CHANGED! error ===<br />
AWS recycles hostnames, so removing the offending key from $HOME/.ssh/known_hosts will remove the warning. You can find the line to remove by finding the line in the output that says<br />
Offending key in /path/to/hosts/known_hosts:2<br />
Where 2 is the line number of the key that can be deleted. Just remove that line, save the file, and try again.<br />
<br />
=== Why is my notebook hanging? ===<br />
There are a couple common causes for this:<br />
<br />
1. Currently, our Spark notebooks can only run a single Python kernel at a time. If you open multiple notebooks on the same cluster and try to run both, the second notebook will hang. Be sure to close notebooks using "Close and Halt" under the "File" dropdown.<br />
<br />
2. The connection from PySpark to the Spark driver might be lost. Unfortunately the best way to recover from this for the moment seems to be spinning up a new cluster.<br />
<br />
3. Canceling execution of a notebook cell doesn't cancel any spark jobs that might be running in the background. If your spark commands seem to be hanging, try running `sc.cancelAllJobs()`.<br />
<br />
=== How can I keep running after closing the notebook? ===<br />
For long-running computation, it might be nice to close the notebook (and the ssh session) and look at the results later. Unfortunately, '''all cell output will be lost when a notebook is closed''' (for the running cell). To alleviate this, there are a few options:<br />
<br />
1. Have everything output to a variable. These values should still be available when you reconnect.<br />
<br />
2. Put %%capture at the beginning of the cell to store all output. [https://ipython.org/ipython-doc/3/interactive/magics.html#cellmagic-capture See the documentation].<br />
<br />
=== How do I load an external library into the cluster? ===<br />
Assuming you've got a url for the repo, you can create an egg for it this way:<br />
<br />
import sys<br />
import os<br />
!git clone <repo url> && cd <repo-name> && python setup.py bdist_egg<br />
sc.addPyFile('<repo-name>/dist/my-egg-file.egg')<br />
sys.path.append(os.path.join(os.getcwd(), '<repo-name>/dist/my-egg-file.egg'))<br />
<br />
Alternately, you could just create that egg locally, upload it to a web server, then download and install it:<br />
<br />
import requests<br />
import sys<br />
import os<br />
r = requests.get('<url-to-my-egg-file>')<br />
with open('mylibrary.egg', 'wb') as f:<br />
f.write(r.content)<br />
sc.addPyFile('mylibrary.egg')<br />
sys.path.append(os.path.join(os.getcwd(), 'mylibrary.egg'))<br />
<br />
You will want to do this '''before''' you load the library. If the library is already loaded, restart the kernel in the ipython notebook.</div>Wlachhttps://wiki.mozilla.org/index.php?title=EngineeringProductivity/Meetings/2017-03-06&diff=1164730EngineeringProductivity/Meetings/2017-03-062017-03-06T17:53:26Z<p>Wlach: /* Notices, Highlights, Roundtable */ treeherder + graphql</p>
<hr />
<div>= Notices, Highlights, Roundtable =<br />
* Significant Contributions<br />
** deepasrijit1105 added layout/tools/reftest to the flake8 linter in {{bug|1280572}}<br />
** Sebastin Santy [:seban] doing some awesome work prototyping a GraphQL interface to treeherder ({{bug|1332457}})<br />
* Feeback about R&P Monthly e-mail<br />
* OKR's<br />
** [https://docs.google.com/document/d/1jY7-bCR-ra7jnC06YvGySe_Fqb8lw3dyizNOYPluyOk/edit 2017 OKR's]<br />
** [https://mana.mozilla.org/wiki/display/PO/Release+And+Productivity+Planning%2C+Goals%2C+and+Priorities R&P Planning Mana]<br />
*** Datacenter Migration - yellow ([https://app.smartsheet.com/b/home?lx=tU6FQRZ869uQsEHyvi5KLg smartsheet])<br />
*** TaskCluster OSX - green ([https://docs.google.com/spreadsheets/d/1AfLwPUzeuNKO_1g_Gw8Fs6bTmyT8n7Dtr3ppHpo05i0/edit#gid=1590579863 status])<br />
*** Quantum Support - green<br />
***# short-term VCS sync in place; working on longer-term plan<br />
***# sccache2 for rust on track; should land by end of month<br />
***# Rust code coverage viable; [https://docs.google.com/document/d/1dOWi18qrudwaOThNAYoCMS3e9LzhxGUiMLLrQ_WVR9w/edit#heading=h.padkvhjjx5pb vetting the MVP]<br />
*** Security - green<br />
***# Autoland - green<br />
***# Bugzilla ES Cluster - migrating to ES 1.7, hopefully<br />
*** Project Dawn (get rid of Aurora) - [https://docs.google.com/document/d/1GDJ4lgQ9TF5pV4uWrnGfXRGoLOfquR1GVYq_mawTFCY/edit project plan] being defined<br />
*** Intermittent Oranges - green<br />
*** Block Bad Commits - yellow, need more detailed project plan<br />
*** TaskCluster Linux - green, Talos for TC linux starting soon<br />
*** TaskCluster Windows - no work for Eng Prod atm<br />
*** Release Control Center - no work for Eng Prod<br />
<br />
= Newsgroup and Blog Posts =<br />
* [ahal] [https://ahal.ca/blog/2017/fuzzy-try-chooser/ A course of action for replacing try syntax]<br />
* [ahal] [https://ahal.ca/blog/2017/mercurial-absorb/ Absorbing changes into a commit series with Mercurial]<br />
* [janx] [https://janitor.technology/blog/ Janitor News #7; Janitor Alpha 0.0.7 is Live]<br />
<br />
= Goal Updates =<br />
<br />
= Other Project Updates =<br />
<br />
= Holidays and Trips =<br />
<br />
= Misc =<br />
* Today is merge day - R52 (tomorrow), B53, A54, N55</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode17&diff=1164711Community:SummerOfCode172017-03-06T16:55:33Z<p>Wlach: /* Automation & Tools */ remove perfherder, already have an applicant in mind</p>
<hr />
<div>This is Mozilla's list of green-lit project proposals for the 2017 Google Summer of Code. <br />
<br />
<b>Are you a student looking to apply to GSoC with Mozilla?</b> You're in the right place. This page lists all the confirmed [[SummerOfCode|Google Summer of Code]] projects. New suggestions can be made on [[Community:SummerOfCode17:Brainstorming|the Brainstorming page]]. Do not edit this page yourself; contact Mike Hoye or Florian for edits.<br />
<br />
If you're interested in participating in Mozilla's GSoC program, you can choose from the list below, '''but you do not have to'''. You can submit a proposal for your own idea. You should look at the [[Community:SummerOfCode17:Brainstorming|guidelines]], though, and discuss your ideas or application in the #introduction channel on IRC.Mozilla.org. This is important, as GSoC projects '''must have''' a supporting member of the Mozilla community to evaluate and mentor them, named in the application.<br />
<br />
===Application Advice===<br />
<br />
You should do the following:<br />
<br />
* Talk to the mentor. Contact details are on this page; if all you have is a nickname, reach out to them on IRC.<br />
* Read the [https://flossmanuals.net/GSoCStudentGuide/ GSoC Student Guide] and follow its advice.<br />
* Read [http://blog.gerv.net/2006/05/how_not_to_apply_for_summer_of/ How Not To Apply For Summer Of Code] and avoid doing the things listed there.<br />
* Read our examples of good applications: [[SummerOfCode/SampleApplications/1|1]], [[SummerOfCode/SampleApplications/2|2]], [[SummerOfCode/SampleApplications/3|3]].<br />
* Apply on [https://summerofcode.withgoogle.com/ the GSoC site] (note that we have an [[SummerOfCode/ApplicationTemplate|application template]]).<br />
* It is entirely acceptable to apply for 2 or 3 projects, if more than one catches your eye; if the applications are high quality, that can improve your chances. Applying to more than that will seem like spam.<br />
<br />
Questions about individual projects are best addressed to the potential mentor of that project. These should be listed in the table below. If you want to contact a mentor and contact details are not here, ask people in the #introduction channel on IRC: irc://irc.mozilla.org/#introduction. If you have questions of any other sort, send mail to [mailto:mhoye@mozilla.com Mike Hoye]. He will try and respond promptly and direct your questions to the right person. <br />
<br />
===Project List===<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
== Mozilla Platform (Gecko) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
| Livelog Proxy <br />
| Write a server that privileged-clients can open a HTTPS connection to in-order to expose a webhook that http-clients can call.<br />
When normal http-clients access the exposed webhooks the connection will be reverse proxied to the<br />
privileged-clients over their out-going connection.<br />
| golang, github, http, web sockets, node.js<br />
| :jonasfj<br />
| :jonasfj<br />
| This is like to ngrok and localtunnel.me, read up on those. For performance reasons server should be written in golang, with client libraries in golang and node.js.<br />
| <br />
|}<br />
<br />
== Firefox ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| about:telemetry redesign<br />
| about:telemetry is present on all builds of Firefox as a way for users to view the data being stored and sent via Telemetry. It was built before Firefox had multi-process Telemetry and without a clear design. This has resulted in a confusing HTML UI and barely-comprehensible JS.<br />
| webtech (HTML+CSS+JS) and Design<br />
| :chutten<br />
| chutten@mozilla.com<br />
| [[Telemetry/GSoC17:AboutTelemetry|Some information]]<br />
|-<br />
| Boost Session Restore performance<br />
| Session (Re)store is important as a key feature of Firefox. Many people rely on it to re-open a tab from the past or recover from an unfortunate power outage. But making it blazing fast has not been our primary focus, until now. Your goal will be to help us make restoring any session snappy and blazing fast. Expect to learn a lot about the Firefox internals and interact with many core engineers during the SoC course.<br />
| JavaScript<br />
| :mikedeboer<br />
| :mikedeboer, :dao<br />
|<br />
|}<br />
<br />
== Firefox for Android ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Implement WebExtension APIs<br />
| [https://developer.mozilla.org/en-US/Add-ons/WebExtensions WebExtensions] are a cross-browser system for developing browser add-ons. [http://arewewebextensionsyet.com/ Not all APIs are supported] by Firefox for Android yet.<br />
The goal of this project is to:<br />
* Implement some of the missing APIs ([https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus contextMenus], [https://developer.chrome.com/extensions/browsingData browsingData], [identity?], [proxy?])<br />
* Add automated test for those APIs<br />
* Improve the tooling for building WebExtensions for Firefox for Android.<br />
| Java, JavaScript, Android<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
|}<br />
<br />
== Automation & Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! JS static analysis <br />
! Bring some static analysis in our Firefox Javascript code<br />
! Javascript experience, FLOW(?)<br />
! [mailto:s@mozilla.com Sylvestre]<br />
! [mailto:s@mozilla.com Sylvestre] <br />
! This project aims to evaluate and integrate more static analysis for Javascript code into some key sections of the Firefox code. We will focus on deploying [https://github.com/facebook/flow FLOW] on some part of the code as a proof of concept. Depending on the results, this will be extended to more components and/or integration in the developer workflow.<br />
|-<br />
! C++ static analysis <br />
! Add new checkers specific to our base code<br />
! Strong C++ experience, clang<br />
! [mailto:s@mozilla.com Sylvestre]<br />
! [mailto:andi@mozilla.com Andi]<br />
! In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns).<br />
|-<br />
! JSON in Sqlite<br />
! Query JSON Documents stored in Sqlite<br />
! Database, SQL, Python <br />
! [mailto:klahnakoski@mozilla.com Kyle Lahnakoski]<br />
! [mailto:klahnakoski@mozilla.com Kyle Lahnakoski]<br />
! [https://github.com/klahnakoski/JSONQueryExpressionTests/blob/master/docs/GSOC%20Proposal.md Details]<br />
|-<br />
! Regression localization from stack traces<br />
! Tool to blame a patch for causing a newly introduced crash.<br />
! C++, Python<br />
! [mailto:marco@mozilla.com Marco]<br />
! [mailto:marco@mozilla.com Marco]<br />
! The project would involve creating a tool that analyzes a set of stack traces, performing the intersection with the Firefox call graph (which can be generated using Clang) and a set of recently committed patches.<br />
|-<br />
! Improvements for crash clustering<br />
! Build a tool to improve crash clustering, currently based on the top method of the stack trace.<br />
! Python, C++<br />
! [mailto:marco@mozilla.com Marco]<br />
! [mailto:marco@mozilla.com Marco]<br />
! The project would involve finalizing https://github.com/marco-c/crashsimilarity; testing it more thoroughly; putting it in production.<br />
|}<br />
<br />
== Mozilla Developer Network ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| MDN Documentation Tester WebExtension<br />
| Rewrite the [https://github.com/Elchi3/mdn-doc-tests MDN doc tester add-on] as a (well-tested, modern, documented) WebExtension that runs in Firefox, Chrome, Opera, and possibly Edge. Ideally reach feature parity with the existing Add-on SDK add-on and if time allows work on new test ideas that run against the MDN documentation. <br />
| Add-On/WebExtension experience (JavaScript, HTML, CSS)<br />
| Florian Scholz<br />
| Florian Scholz<br />
|}<br />
<br />
<br />
== Release Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Improve Balrog's Admin API<br />
| Make Balrog's Admin API simpler and easier for clients to work with by moving to a [http://swagger.io/ Swagger]-based approach.<br />
| python,rest apis<br />
| Ben Hearsum (bhearsum@mozilla.com)<br />
| Ben Hearsum (bhearsum@mozilla.com)<br />
|}<br />
<br />
== Rust ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! Rust + WebAssembly showcase<br />
! Rust and WebAssembly are going to be a great pair. Design and implement a simple and attractive web application, in Rust, that demonstrates the power of Rust on the web. Make fixes to upstream projects as necessary. Write a blog post about it.<br />
! Rust, web development<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson], [https://github.com/badboy badboy]<br />
! cc https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-emscripten/7627. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-1-md<br />
|-<br />
! Rust dashboard updates<br />
! Update rusty-dash.com to include additional metrics important to the project. This tool is vital to the day-to-day management of Rust, but it needs some dedicated attention to fulfill its promise.<br />
! Rust<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson]<br />
! Follow on work to https://internals.rust-lang.org/t/the-rust-project-needs-much-better-visibility-into-important-metrics/3367<br />
|-<br />
! Rust reproducible builds<br />
! Make the Rust build process produce the identical binaries when run with identical configurations. This improves the security of the Rust ecosystem by allowing others to<br />
double-check the official Rust builds<br />
! Rust, compilers, systems programming<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson], [https://github.com/michaelwoerister mw]<br />
! https://internals.rust-lang.org/t/verifying-rustc-releases-with-reproducible-builds/4502. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-2-md<br />
|-<br />
! Abstract the Rust standard library<br />
! The Rust standard library is very portable, but can be very, very portable. Refactor the<br />
standard library to pull out a platform abstraction layer.<br />
! Moderate Rust experience<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson]<br />
! https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301<br />
|-<br />
! Rust cross-platform showcase project<br />
! Rust is very portable. Create a single showcase demo project that compiles for Windows, OS X, Linux, Android, iOS, wasm, and microcontrollers. Use real crates to accomplish some real task. Set up CI for all platforms on Travis. To be used as a teaching tool and for regression testing. Write a blog post.<br />
! Moderate Rust experience<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson]<br />
! <br />
|-<br />
! rustc micro-optimization bonanza<br />
! Just go hog wild finding microoptimizations in rustc. Write a blog post bragging about it.<br />
! Performance optimization<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson]<br />
! <br />
|-<br />
! Rust-specific benchmark suite<br />
! Today Rust uses perf.rust-lang.org to track _compile time_ performance, but nothing to track _runtime_ performance. Work with the Rust developers to create a benchmark suite specific to Rust.<br />
! Programming<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson]<br />
! <br />
|-<br />
! Rust crate semver compatibility tool<br />
! Rust libraries follow the semver spec for indicating API compatibility, but conformance to semver is not enforced in any way - it is up to crate authors to guarantee their crates can be upgraded correctly. With Rust's strong type system it should be possible to mechanically check whether crates<br />
are obeying semver. This would be a huge boon to the stability of the Rust ecosystem.<br />
! Static analysis<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/brson/ brson], [https://github.com/badboy/ badboy]<br />
! https://users.rust-lang.org/t/warnings-for-breaking-semver/1415 https://users.rust-lang.org/t/signature-based-api-comparison/2377 . Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-3-md<br />
|-<br />
! Reconstruct the Rust bootstrap chain<br />
! Rust is a self-hosted compiler, originally bootstrapped from OCaml, then self-bootstrapped several hundred times over the years. Today there is only one Rust 'lineage' of any note, the official compiler, but with compilers diversity of implementation is a security issue. If another party could replicate the<br />
Rust compiler it would provide assurance that the official isn't compromised by, or can recover from, a "trusting trust" attack. Although the historical information needed to rebootstrap Rust is mostly complete, it is not trivial to do. Construct a script that can reconstruct the modern Rust compiler from the original OCaml implementation, run it to completion, write a blog post.<br />
! Scripting, build systems<br />
! [https://github.com/brson/ brson]<br />
! [https://github.com/alexcrichton/ acrichto]<br />
! <br />
|-<br />
|}<br />
<br />
== Servo ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Off-main thread HTML parsing<br />
| [https://github.com/servo/servo/wiki/Off-main-thread-HTML-parsing-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [mailto:aramine@mozilla.com Anthony Ramine (nox)]<br />
|<br />
|-<br />
| Extend ServiceWorker implementation<br />
| [https://github.com/servo/servo/wiki/More-ServiceWorker-support-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| <br />
|}<br />
<br />
== Security Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Tor Experiment<br />
| Create a an experimental extension that bundles and launches Tor. Benchmark and report the amount of data transferred, time-to-first-byte, bandwidth differentials.<br />
| Add-On creation experience, Programming in C/Javascript.<br />
| Tom Ritter<br />
| Tom Ritter<br />
|-<br />
| StopTrackware.org<br />
| Create a clearinghouse of trackware lists like StopBadware.org, by creating a Track-the-trackers add-on that uses data [http://www2016.net/proceedings/proceedings/p121.pdf "safeness" approach like Cliqz]<br />
| Add-On creation experience (JavaScript, HTML, CSS, Networking), Cloud Service experience (python or node.js)<br />
| Luke Crouch<br />
| Luke Crouch<br />
|<br />
|}<br />
<br />
== Localization ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Design screenshot-based localization interface in [https://pontoon.mozilla.org/ Pontoon]<br />
| We'd like to explore the feasibility of screenshot-based localization process in [https://pontoon.mozilla.org/ Pontoon] - Mozilla's translation tool. To provide more translation context, each string will be accompanied by a screenshot showing how the string is used in the application. Additionally, localizers will have the ability to navigate and filter strings by screenshots, as well as preview their translations in the localized screenshots.<br />
| Web standards and Design. Django is a plus.<br />
| [[User:Mathjazz|:mathjazz]]<br />
| [[User:Mathjazz|:mathjazz]]<br />
| <br />
|}<br />
<br />
== Security Assurance ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|<br />
TLS 1.3 scanning in Mozilla TLS Observatory<br />
|<br />
[https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to improve the service to support scanning TLS 1.3 enabled endpoints, either by improving the [https://github.com/mozilla/cipherscan existing scanner] or writing an entirely new scanner.<br />
|<br />
Programming skills in C, Bash and Go. Strong understanding of TLS and micro-services architectures.<br />
|<br />
Julien Vehent<br />
|<br />
Julien Vehent<br />
|<br />
None<br />
|}<br />
<br />
== Mozilla Science Lab ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| science.mozilla.org phase 2<br />
| Start phase 2 of the Mozilla Science Lab website by implementing<br />
* [https://github.com/mozilla/science.mozilla.org/issues/135 github authentication], <br />
* [https://github.com/mozilla/science.mozilla.org/issues/311 members homepage],<br />
* [https://github.com/mozilla/science.mozilla.org/issues/536 improve blog post readability], and add the<br />
* [https://github.com/mozilla/science.mozilla.org/issues/518 ability to join projects]<br />
| js, html, css, react, django, python<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/alanmoo/ alanmoo]<br />
| Detailed list and breakdown of all phase 2 work (only the tasks mentioned in details are expected to be completed by the student):<br />
* Phase 2.1 https://github.com/mozilla/science.mozilla.org/milestone/11<br />
* Phase 2.2 https://github.com/mozilla/science.mozilla.org/milestone/12 <br />
|-<br />
| Paper Badger - badgr migration & publisher integration<br />
| * Issuing badges to credit authors for their work on academic papers.<br />
* In the final steps of our badgekit to Badgr migration, add tests to make sure we have feature parity and make the switch to Badgr in production.<br />
* Implement a [https://github.com/mozillascience/PaperBadger/issues/160 journal submission system integration] with paper badger. This way, journals won't have to manually enter new badges in our system.<br />
| js, Badgr, APIs, authentication, node.js, react<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| github: https://github.com/mozillascience/paperbadger<br />
|-<br />
| Study Group participation visualization<br />
| [https://github.com/mozillascience/studyGroup study groups] meet all over the world, running regular events share skills, co-work and create community around open science. Right now there's no easy way to see all the study group events.<br />
* Write a script to collect data for all study group events happening world wide. Each study group is a [https://github.com/mozillascience/studyGroup fork of this template repo]. Each event is an issue in the repo ([https://github.com/UofTCoders/Events/issues example of events as issues]). [https://github.com/mozillascience/studyGroup/blob/gh-pages/scripts/updateCalendar.py here is a script] that collects all events for a specific repo. Please update this to collect data from all forks.<br />
* Visualize this data in a way that shows study group activity all around the world! viz: calendar, map<br />
| js, html, css, python (or another scripting language), data visualization<br />
| [https://mozillians.org/en-US/u/zannah/ Zannah]<br />
| [https://mozillians.org/en-US/u/aurelia/ Aurelia]<br />
| main study group repo: https://github.com/mozillascience/studyGroup<br />
|}<br />
<br />
<br />
== Thimble ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Address stability issues within Thimble<br />
| Main task: Get Thimble to write directly to AWS instead of writing to a database first.<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| Full list of issues affecting thimble's stability: https://github.com/mozilla/thimble.mozilla.org/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20bug%22%20-label%3A%22assigned%20to%20contributor%22<br />
|-<br />
| Integrating VR capabilities into Thimble<br />
| Goal: Thimble makes it easy to create multi user VR experiences, allowing the whole classroom to take virtual field trips through student projects.<br />
This is made possible by doing any of these:<br />
* Integration of the A-frame visual inspector, which gives the user lots of control over an object properties in a custom UI<br />
* Custom Quick-Edit UI for different types of objects<br />
* Develop large, visual menu of 3D objects make it easy to build interesting scenes quickly<br />
* A wide range of starter projects<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This work would be done in parallel with work done by [https://cdot.senecacollege.ca/ CDOT] on making Thimble a VR experience building tool<br />
thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|-<br />
| Thimble and Remote Mentorship<br />
| Goal: Use together.js to enable a multi-user environment on Thimble that is tailored to work as a virtual teaching environment for HTML/CSS/JS<br />
Details:<br />
* Fixing issues in together.js to make it functional<br />
* working on ways to sync state between Thimble's editor and the remote Thimble editor(s<br />
* Syncing filesystems across Thimble editors with WebRTC<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode17:Brainstorming&diff=1164710Community:SummerOfCode17:Brainstorming2017-03-06T16:54:57Z<p>Wlach: remove perfherder project, already have an applicant in mind</p>
<hr />
<div>Mozilla community members - submit proposals here for 2017 Google Summer of Code projects with Mozilla. (If this page looks empty, it's because accepted ideas have already been transferred to the [[Community:SummerOfCode17|official list]].) '''The''' absolute last '''deadline for submitting ideas''' in time to help us get accepted by Google '''is February 9th'''.<br />
<br />
<b>Are you a student looking to apply to SoC with Mozilla?</b> Your first stop should be the [[Community:SummerOfCode17|official list of ideas]]. This page is full of weird and whacky ideas, some of which are still on here for a reason - it could be that they are not properly defined, the wrong size, or don't have a mentor. That makes them less likely to get accepted. You <i>can</i>, of course, also submit your own ideas - you don't have to put an idea on this page and get it 'made official' in order to send in a proposal for it.<br />
<br />
==How To Write A Good Project Proposal==<br />
<br />
Before adding an proposal to this list, please consider the following:<br />
<br />
* '''Be specific'''. It's hard to understand the impact of, or the size of, vague proposals.<br />
* '''Consider size'''. The student has eight weeks to design, code, test and document the proposal. It needs to fill, but not overfill, that time.<br />
* '''Do your research'''. Support the idea with well-researched links.<br />
* '''Don't morph other people's ideas'''. If you have a related idea, place it next to the existing one, or add a comment. <br />
* '''Insert only your own name into the Mentor column''', and then only if you are willing to take on the responsibility. If you think the SoC admins won't know who you are, leave contact details.<br />
* '''Check back regularly'''. The administrators may have questions about your idea that you will need to answer.<br />
* '''Know when to give up'''. If you've added the same idea for the last three years and it hasn't made it to the official page, perhaps you can predict what will happen this time.<br />
<br />
==Suggestion List==<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
== Mozilla Platform (Gecko) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| TaskCluster JSON Parameterization<br />
| Build a powerful JSON parameterization system and use it everywhere<br />
| JavaScript, Python<br />
| :dustin<br />
| :dustin<br />
| The json-e language supports complex transformations of JSON data. The project involves completing the specification, implementation (in two languages), and documentation of this language, then using it to support Gecko action and decision task and users of taskcluster-github and taskcluster-hooks. Success here means that the language is complete and in active use in at least one of the listed contexts.<br />
|-<br />
| Livelog Proxy <br />
| Write a server that privileged-clients can open a HTTPS connection to in-order to expose a webhook that http-clients can call.<br />
When normal http-clients access the exposed webhooks the connection will be reverse proxied to the<br />
privileged-clients over their out-going connection.<br />
| golang, github, http, web sockets, node.js<br />
| :jonasfj<br />
| :jonasfj<br />
| This is like to ngrok and localtunnel.me, read up on those. For performance reasons server should be written in golang, with client libraries in golang and node.js.<br />
| <br />
|}<br />
<br />
== Firefox ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| about:telemetry redesign<br />
| about:telemetry is present on all builds of Firefox as a way for users to view the data being stored and sent via Telemetry. It was built before Firefox had multi-process Telemetry and without a clear design. This has resulted in a confusing HTML UI and barely-comprehensible JS.<br />
| webtech (HTML+CSS+JS) and Design<br />
| :chutten<br />
| :chutten<br />
| <br />
|-<br />
| Boost Session Restore performance<br />
| Session (Re)store is important as a key feature of Firefox. Many people rely on it to re-open a tab from the past or recover from an unfortunate power outage. But making it blazing fast has not been our primary focus, until now. Your goal will be to help us make restoring any session snappy and blazing fast. Expect to learn a lot about the Firefox internals and interact with many core engineers during the SoC course.<br />
| JavaScript<br />
| :mikedeboer<br />
| :mikedeboer, :dao<br />
|<br />
|}<br />
<br />
== Firefox Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Developer Tools for Designers<br />
| In Firefox Developer Tools, we're aiming to implement visual tools that will help developers and designers edit CSS properties visually. Since CSS is very declarative by nature, we can provide tools that are commonly found in designer tools in order to bridge the learning gap of new developers and designers learning CSS and allow them to edit complex CSS features such as CSS filters, blend, clip-path, layout, etc without knowing what the properties are. Going beyond editing CSS, we also need to think about how we can visually inspect the CSS style transformations that can impact the layout. Finally, we are looking for ways to better export and see the changes made in the Inspector tool so that we can take the work we have done in the Developer Tools and apply that to your local source files.<br />
| JavaScript, HTML, CSS, React, Redux<br />
| [[User:Gl|:gl]]<br />
| [[User:Gl|:gl]]<br />
| Starting meta bug for inspiration [https://bugzilla.mozilla.org/show_bug.cgi?id=1258390 Bug 1258390]<br />
|}<br />
<br />
== Firefox for Android ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Implement WebExtension APIs<br />
| [https://developer.mozilla.org/en-US/Add-ons/WebExtensions WebExtensions] are a cross-browser system for developing browser add-ons. [http://arewewebextensionsyet.com/ Not all APIs are supported] by Firefox for Android yet.<br />
The goal of this project is to:<br />
* Implement some of the missing APIs ([https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus contextMenus], [https://developer.chrome.com/extensions/browsingData browsingData], [identity?], [proxy?])<br />
* Add automated test for those APIs<br />
* Improve the tooling for building WebExtensions for Firefox for Android.<br />
| Java, JavaScript, Android<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
|}<br />
<br />
== Thunderbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Instantbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
| Matrix protocol support<br />
| [http://matrix.org/ Matrix] is an open, decentralized protocol for instant messaging (and more!) It has bridges to many other networks and protocol, e.g. IRC, Slack, and more. Initial support for Matrix was added in {{bug|1199855}}, but there's a lot to do still:<br />
<br />
* Support more features from the Matrix SDK (room topics, typing notifications, read receipts, etc.)<br />
* Support one-on-one conversations.<br />
* Add tests specific to Matrix.<br />
* Improve the Matrix JS-SDK that Instantbird and Thunderbird depend on.<br />
* Improving and expanding shared code and APIs used by all JavaScript protocol plugins (IRC, XMPP, Yahoo and Twitter).<br />
* Improving documentation of the process for adding a protocol to Instantbird/Thunderbird.<br />
* Using the Matrix protocol on a day-to-day basis to dog-food the code and see what else is missing.<br />
| JavaScript, XPCOM, experience using HTTP APIs<br />
| Patrick Cloke [:clokep]<br />
| [mailto:clokep@gmail.com clokep]<br />
| If you're interested in supporting a different protocol, please come talk to us. Protocols we're interested in: Matrix, Google Hangouts, Facebook, Bonjour, TextSecure/Signal, Telegram, or others!<br />
|}<br />
<br />
== Open Innovation (Campus Clubs) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Measuring the Development of an Open Source Contributor<br />
| We are looking for a passionate developer to work with the GitHub API to build a platform for measuring contributors performance/engagement across multiple open source projects over an extended time period.<br />
| Javascript, javascript framework, a sequel database, HTML, GitHub API <br />
| [https://mozillians.org/en-US/u/lucyeoh/ :Lucy]<br />
| [https://mozillians.org/en-US/u/bacharakis/ :Christos]<br />
| For questions or to express interest in this project please leave a comment in **[https://github.com/mozilla/Campus-Program/issues/363 this issue]** <br />
|-<br />
|}<br />
<br />
== Calendar ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== SeaMonkey ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Bugzilla ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Firefox Support (SUMO) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== QA ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Automation & Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! JS static analysis <br />
! Bring some static analysis in our Firefox Javascript code<br />
! Javascript experience, FLOW(?)<br />
! Sylvestre<br />
! same <br />
! This project aims to evaluate and integrate more static analysis for Javascript code into some key sections of the Firefox code. We will focus on deploying [https://github.com/facebook/flow FLOW] on some part of the code as a proof of concept. Depending on the results, this will be extended to more components and/or integration in the developer workflow.<br />
|-<br />
! C++ static analysis <br />
! Add new checkers specific to our base code<br />
! Strong C++ experience, clang<br />
! Sylvestre<br />
! Andi<br />
! In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns).<br />
|-<br />
! JSON in Sqlite<br />
! Query JSON Documents stored in Sqlite<br />
! Database, SQL, Python <br />
! Kyle Lahnakoski<br />
! Kyle Lahnakoski<br />
! [https://github.com/klahnakoski/JSONQueryExpressionTests/blob/master/docs/GSOC%20Proposal.md Details]<br />
|-<br />
! Regression localization from stack traces<br />
! Tool to blame a patch for causing a newly introduced crash.<br />
! C++, Python<br />
! Marco<br />
! Marco<br />
! The project would involve creating a tool that analyzes a set of stack traces, performing the intersection with the Firefox call graph (which can be generated using Clang) and a set of recently committed patches.<br />
|-<br />
! Improvements for crash clustering<br />
! Build a tool to improve crash clustering, currently based on the top method of the stack trace.<br />
! Python, C++<br />
! Marco<br />
! Marco<br />
! The project would involve finalizing https://github.com/marco-c/crashsimilarity; testing it more thoroughly; putting it in production.<br />
|}<br />
<br />
== Documentation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Mozilla Developer Network ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| MDN Documentation Tester WebExtension<br />
| Rewrite the [https://github.com/Elchi3/mdn-doc-tests MDN doc tester add-on] as a (well-tested, modern, documented) WebExtension that runs in Firefox, Chrome, Opera, and possibly Edge. Ideally reach feature parity with the existing Add-on SDK add-on and if time allows work on new test ideas that run against the MDN documentation. <br />
| Add-On/WebExtension experience (JavaScript, HTML, CSS)<br />
| Florian Scholz<br />
| Florian Scholz<br />
|}<br />
<br />
== Mozilla IT and Infrastructure ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Sync / Services ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Add-on SDK ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Foundation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Curriculum database for Learning site<br />
| Curriculum is currently accessed via the Teaching Activities page of the learning site (https://learning.mozilla.org/en-US/activities ). It's a pretty static experience. Users can browse through a list of modules, but there is no search capability. We have a prototype for a more interactive UX that allows users to search and filter curriculum modules and activities: https://flukeout.github.io/curriculum-search-react/. This project would require implementing the front-end, as well as wiring up the backend database (for which the model has been defined, but the data has not been migrated).<br />
| js, html, css, django, python<br />
| [https://mozillians.org/en-US/u/hannah/ hannah]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This would be a big win for the curriculum team and for the community members who make good use of our learning/teaching materials. The database could eventually be extended beyond the learning site and support other parts of the network as well.<br />
|}<br />
<br />
== OpenArt ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Release Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Improve Balrog's Admin API<br />
| Make Balrog's Admin API simpler and easier for clients to work with by moving to a [http://swagger.io/ Swagger]-based approach.<br />
| python,rest apis<br />
| Ben Hearsum<br />
| Ben Hearsum<br />
|}<br />
<br />
== Emscripten ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Rust ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! Rust + WebAssembly showcase<br />
! Rust and WebAssembly are going to be a great pair. Design and implement a simple and attractive web application, in Rust, that demonstrates the power of Rust on the web. Make fixes to upstream projects as necessary. Write a blog post about it.<br />
! Rust, web development<br />
! brson<br />
! brson, badboy<br />
! cc https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-emscripten/7627. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-1-md<br />
|-<br />
! Rust dashboard updates<br />
! Update rusty-dash.com to include additional metrics important to the project. This tool is vital to the day-to-day management of Rust, but it needs some dedicated attention to fulfill its promise.<br />
! Rust<br />
! brson<br />
! brson, aturon?, dikaiosune?<br />
! Follow on work to https://internals.rust-lang.org/t/the-rust-project-needs-much-better-visibility-into-important-metrics/3367<br />
|-<br />
! Rust reproducible builds<br />
! Make the Rust build process produce the identical binaries when run with identical configurations. This improves the security of the Rust ecosystem by allowing others to<br />
double-check the official Rust builds<br />
! Rust, compilers, systems programming<br />
! brson<br />
! brson, mw, Manishearth<br />
! https://internals.rust-lang.org/t/verifying-rustc-releases-with-reproducible-builds/4502. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-2-md<br />
|-<br />
! Abstract the Rust standard library<br />
! The Rust standard library is very portable, but can be very, very portable. Refactor the<br />
standard library to pull out a platform abstraction layer.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301<br />
|-<br />
! Rust cross-platform showcase project<br />
! Rust is very portable. Create a single showcase demo project that compiles for Windows, OS X, Linux, Android, iOS, wasm, and microcontrollers. Use real crates to accomplish some real task. Set up CI for all platforms on Travis. To be used as a teaching tool and for regression testing. Write a blog post.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! rustc micro-optimization bonanza<br />
! Just go hog wild finding microoptimizations in rustc. Write a blog post bragging about it.<br />
! Performance optimization<br />
! brson<br />
! brson, nnethercote?<br />
! <br />
|-<br />
! Rust-specific benchmark suite<br />
! Today Rust uses perf.rust-lang.org to track _compile time_ performance, but nothing to track _runtime_ performance. Work with the Rust developers to create a benchmark suite specific to Rust.<br />
! Programming<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! Rust crate semver compatibility tool<br />
! Rust libraries follow the semver spec for indicating API compatibility, but conformance to semver is not enforced in any way - it is up to crate authors to guarantee their crates can be upgraded correctly. With Rust's strong type system it should be possible to mechanically check whether crates<br />
are obeying semver. This would be a huge boon to the stability of the Rust ecosystem.<br />
! Static analysis<br />
! brson<br />
! brson, badboy<br />
! https://users.rust-lang.org/t/warnings-for-breaking-semver/1415 https://users.rust-lang.org/t/signature-based-api-comparison/2377 . Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-3-md<br />
|-<br />
! Reconstruct the Rust bootstrap chain<br />
! Rust is a self-hosted compiler, originally bootstrapped from OCaml, then self-bootstrapped several hundred times over the years. Today there is only one Rust 'lineage' of any note, the official compiler, but with compilers diversity of implementation is a security issue. If another party could replicate the<br />
Rust compiler it would provide assurance that the official isn't compromised by, or can recover from, a "trusting trust" attack. Although the historical information needed to rebootstrap Rust is mostly complete, it is not trivial to do. Construct a script that can reconstruct the modern Rust compiler from the original OCaml implementation, run it to completion, write a blog post.<br />
! Scripting, build systems<br />
! brson<br />
! acrichto<br />
! <br />
|-<br />
|}<br />
<br />
== Servo ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Off-main thread HTML parsing<br />
| [https://github.com/servo/servo/wiki/Off-main-thread-HTML-parsing-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| Anthony Ramine (nox)<br />
|<br />
|-<br />
| Extend ServiceWorker implementation<br />
| [https://github.com/servo/servo/wiki/More-ServiceWorker-support-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| <br />
|}<br />
<br />
== Security Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Tor Experiment<br />
| Create a an experimental extension that bundles and launches Tor. Benchmark and report the amount of data transferred, time-to-first-byte, bandwidth differentials.<br />
| Add-On creation experience, Programming in C/Javascript.<br />
| Tom Ritter<br />
| Tom Ritter<br />
|-<br />
| StopTrackware.org<br />
| Create a clearinghouse of trackware lists like StopBadware.org, by creating a Track-the-trackers add-on that uses data [http://www2016.net/proceedings/proceedings/p121.pdf "safeness" approach like Cliqz]<br />
| Add-On creation experience (JavaScript, HTML, CSS, Networking), Cloud Service experience (python or node.js)<br />
| Luke Crouch<br />
| Luke Crouch<br />
|<br />
|}<br />
<br />
== Localization ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Design screenshot-based localization interface in [https://pontoon.mozilla.org/ Pontoon]<br />
| We'd like to explore the feasibility of screenshot-based localization process in [https://pontoon.mozilla.org/ Pontoon] - Mozilla's translation tool. To provide more translation context, each string will be accompanied by a screenshot showing how the string is used in the application. Additionally, localizers will have the ability to navigate and filter strings by screenshots, as well as preview their translations in the localized screenshots.<br />
| Web standards and Design. Django is a plus.<br />
| [[User:Mathjazz|:mathjazz]]<br />
| [[User:Mathjazz|:mathjazz]]<br />
| <br />
|}<br />
<br />
== Build system ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Security Assurance ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|<br />
TLS 1.3 scanning in Mozilla TLS Observatory<br />
|<br />
[https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to improve the service to support scanning TLS 1.3 enabled endpoints, either by improving the [https://github.com/mozilla/cipherscan existing scanner] or writing an entirely new scanner.<br />
|<br />
Programming skills in C, Bash and Go. Strong understanding of TLS and micro-services architectures.<br />
|<br />
Julien Vehent<br />
|<br />
Julien Vehent<br />
|<br />
None<br />
|}<br />
<br />
== WADI ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Mozilla Science Lab ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| science.mozilla.org phase 2<br />
| Start phase 2 of the Mozilla Science Lab website by implementing<br />
* [https://github.com/mozilla/science.mozilla.org/issues/135 github authentication], <br />
* [https://github.com/mozilla/science.mozilla.org/issues/311 members homepage],<br />
* [https://github.com/mozilla/science.mozilla.org/issues/536 improve blog post readability], and add the<br />
* [https://github.com/mozilla/science.mozilla.org/issues/518 ability to join projects]<br />
| js, html, css, react, django, python<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/alanmoo/ alanmoo]<br />
| Detailed list and breakdown of all phase 2 work (only the tasks mentioned in details are expected to be completed by the student):<br />
* Phase 2.1 https://github.com/mozilla/science.mozilla.org/milestone/11<br />
* Phase 2.2 https://github.com/mozilla/science.mozilla.org/milestone/12 <br />
|-<br />
| Paper Badger - badgr migration & publisher integration<br />
| * Issuing badges to credit authors for their work on academic papers.<br />
* In the final steps of our badgekit to Badgr migration, add tests to make sure we have feature parity and make the switch to Badgr in production.<br />
* Implement a [https://github.com/mozillascience/PaperBadger/issues/160 journal submission system integration] with paper badger. This way, journals won't have to manually enter new badges in our system.<br />
| js, Badgr, APIs, authentication, node.js, react<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| github: https://github.com/mozillascience/paperbadger<br />
|-<br />
| Study Group participation visualization<br />
| [https://github.com/mozillascience/studyGroup study groups] meet all over the world, running regular events share skills, co-work and create community around open science. Right now there's no easy way to see all the study group events.<br />
* Write a script to collect data for all study group events happening world wide. Each study group is a [https://github.com/mozillascience/studyGroup fork of this template repo]. Each event is an issue in the repo ([https://github.com/UofTCoders/Events/issues example of events as issues]). [https://github.com/mozillascience/studyGroup/blob/gh-pages/scripts/updateCalendar.py here is a script] that collects all events for a specific repo. Please update this to collect data from all forks.<br />
* Visualize this data in a way that shows study group activity all around the world! viz: calendar, map<br />
| js, html, css, python (or another scripting language), data visualization<br />
| [https://mozillians.org/en-US/u/zannah/ Zannah]<br />
| [https://mozillians.org/en-US/u/aurelia/ Aurelia]<br />
| main study group repo: https://github.com/mozillascience/studyGroup<br />
|}<br />
<br />
<br />
== MozVR ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
== Connected Devices ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
<br />
<br />
== Thimble ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Address stability issues within Thimble<br />
| Main task: Get Thimble to write directly to AWS instead of writing to a database first.<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| Full list of issues affecting thimble's stability: https://github.com/mozilla/thimble.mozilla.org/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20bug%22%20-label%3A%22assigned%20to%20contributor%22<br />
|-<br />
| Integrating VR capabilities into Thimble<br />
| Goal: Thimble makes it easy to create multi user VR experiences, allowing the whole classroom to take virtual field trips through student projects.<br />
This is made possible by doing any of these:<br />
* Integration of the A-frame visual inspector, which gives the user lots of control over an object properties in a custom UI<br />
* Custom Quick-Edit UI for different types of objects<br />
* Develop large, visual menu of 3D objects make it easy to build interesting scenes quickly<br />
* A wide range of starter projects<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This work would be done in parallel with work done by [https://cdot.senecacollege.ca/ CDOT] on making Thimble a VR experience building tool<br />
thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|-<br />
| Thimble and Remote Mentorship<br />
| Goal: Use together.js to enable a multi-user environment on Thimble that is tailored to work as a virtual teaching environment for HTML/CSS/JS<br />
Details:<br />
* Fixing issues in together.js to make it functional<br />
* working on ways to sync state between Thimble's editor and the remote Thimble editor(s<br />
* Syncing filesystems across Thimble editors with WebRTC<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode17:Brainstorming&diff=1164568Community:SummerOfCode17:Brainstorming2017-03-04T00:58:27Z<p>Wlach: /* Automation & Tools */ not actually looking for applicants for graphql project</p>
<hr />
<div>Mozilla community members - submit proposals here for 2017 Google Summer of Code projects with Mozilla. (If this page looks empty, it's because accepted ideas have already been transferred to the [[Community:SummerOfCode17|official list]].) '''The''' absolute last '''deadline for submitting ideas''' in time to help us get accepted by Google '''is February 9th'''.<br />
<br />
<b>Are you a student looking to apply to SoC with Mozilla?</b> Your first stop should be the [[Community:SummerOfCode17|official list of ideas]]. This page is full of weird and whacky ideas, some of which are still on here for a reason - it could be that they are not properly defined, the wrong size, or don't have a mentor. That makes them less likely to get accepted. You <i>can</i>, of course, also submit your own ideas - you don't have to put an idea on this page and get it 'made official' in order to send in a proposal for it.<br />
<br />
==How To Write A Good Project Proposal==<br />
<br />
Before adding an proposal to this list, please consider the following:<br />
<br />
* '''Be specific'''. It's hard to understand the impact of, or the size of, vague proposals.<br />
* '''Consider size'''. The student has eight weeks to design, code, test and document the proposal. It needs to fill, but not overfill, that time.<br />
* '''Do your research'''. Support the idea with well-researched links.<br />
* '''Don't morph other people's ideas'''. If you have a related idea, place it next to the existing one, or add a comment. <br />
* '''Insert only your own name into the Mentor column''', and then only if you are willing to take on the responsibility. If you think the SoC admins won't know who you are, leave contact details.<br />
* '''Check back regularly'''. The administrators may have questions about your idea that you will need to answer.<br />
* '''Know when to give up'''. If you've added the same idea for the last three years and it hasn't made it to the official page, perhaps you can predict what will happen this time.<br />
<br />
==Suggestion List==<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
== Mozilla Platform (Gecko) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| TaskCluster JSON Parameterization<br />
| Build a powerful JSON parameterization system and use it everywhere<br />
| JavaScript, Python<br />
| :dustin<br />
| :dustin<br />
| The json-e language supports complex transformations of JSON data. The project involves completing the specification, implementation (in two languages), and documentation of this language, then using it to support Gecko action and decision task and users of taskcluster-github and taskcluster-hooks. Success here means that the language is complete and in active use in at least one of the listed contexts.<br />
|-<br />
| Livelog Proxy <br />
| Write a server that privileged-clients can open a HTTPS connection to in-order to expose a webhook that http-clients can call.<br />
When normal http-clients access the exposed webhooks the connection will be reverse proxied to the<br />
privileged-clients over their out-going connection.<br />
| golang, github, http, web sockets, node.js<br />
| :jonasfj<br />
| :jonasfj<br />
| This is like to ngrok and localtunnel.me, read up on those. For performance reasons server should be written in golang, with client libraries in golang and node.js.<br />
| <br />
|}<br />
<br />
== Firefox ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| about:telemetry redesign<br />
| about:telemetry is present on all builds of Firefox as a way for users to view the data being stored and sent via Telemetry. It was built before Firefox had multi-process Telemetry and without a clear design. This has resulted in a confusing HTML UI and barely-comprehensible JS.<br />
| webtech (HTML+CSS+JS) and Design<br />
| :chutten<br />
| :chutten<br />
| <br />
|-<br />
| Boost Session Restore performance<br />
| Session (Re)store is important as a key feature of Firefox. Many people rely on it to re-open a tab from the past or recover from an unfortunate power outage. But making it blazing fast has not been our primary focus, until now. Your goal will be to help us make restoring any session snappy and blazing fast. Expect to learn a lot about the Firefox internals and interact with many core engineers during the SoC course.<br />
| JavaScript<br />
| :mikedeboer<br />
| :mikedeboer, :dao<br />
|<br />
|}<br />
<br />
== Firefox Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Developer Tools for Designers<br />
| In Firefox Developer Tools, we're aiming to implement visual tools that will help developers and designers edit CSS properties visually. Since CSS is very declarative by nature, we can provide tools that are commonly found in designer tools in order to bridge the learning gap of new developers and designers learning CSS and allow them to edit complex CSS features such as CSS filters, blend, clip-path, layout, etc without knowing what the properties are. Going beyond editing CSS, we also need to think about how we can visually inspect the CSS style transformations that can impact the layout. Finally, we are looking for ways to better export and see the changes made in the Inspector tool so that we can take the work we have done in the Developer Tools and apply that to your local source files.<br />
| JavaScript, HTML, CSS, React, Redux<br />
| [[User:Gl|:gl]]<br />
| [[User:Gl|:gl]]<br />
| Starting meta bug for inspiration [https://bugzilla.mozilla.org/show_bug.cgi?id=1258390 Bug 1258390]<br />
|}<br />
<br />
== Firefox for Android ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Implement WebExtension APIs<br />
| [https://developer.mozilla.org/en-US/Add-ons/WebExtensions WebExtensions] are a cross-browser system for developing browser add-ons. [http://arewewebextensionsyet.com/ Not all APIs are supported] by Firefox for Android yet.<br />
The goal of this project is to:<br />
* Implement some of the missing APIs ([https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus contextMenus], [https://developer.chrome.com/extensions/browsingData browsingData], [identity?], [proxy?])<br />
* Add automated test for those APIs<br />
* Improve the tooling for building WebExtensions for Firefox for Android.<br />
| Java, JavaScript, Android<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
|}<br />
<br />
== Thunderbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Instantbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
| Matrix protocol support<br />
| [http://matrix.org/ Matrix] is an open, decentralized protocol for instant messaging (and more!) It has bridges to many other networks and protocol, e.g. IRC, Slack, and more. Initial support for Matrix was added in {{bug|1199855}}, but there's a lot to do still:<br />
<br />
* Support more features from the Matrix SDK (room topics, typing notifications, read receipts, etc.)<br />
* Support one-on-one conversations.<br />
* Add tests specific to Matrix.<br />
* Improve the Matrix JS-SDK that Instantbird and Thunderbird depend on.<br />
* Improving and expanding shared code and APIs used by all JavaScript protocol plugins (IRC, XMPP, Yahoo and Twitter).<br />
* Improving documentation of the process for adding a protocol to Instantbird/Thunderbird.<br />
* Using the Matrix protocol on a day-to-day basis to dog-food the code and see what else is missing.<br />
| JavaScript, XPCOM, experience using HTTP APIs<br />
| Patrick Cloke [:clokep]<br />
| [mailto:clokep@gmail.com clokep]<br />
| If you're interested in supporting a different protocol, please come talk to us. Protocols we're interested in: Matrix, Google Hangouts, Facebook, Bonjour, TextSecure/Signal, Telegram, or others!<br />
|}<br />
<br />
== Open Innovation (Campus Clubs) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Measuring the Development of an Open Source Contributor<br />
| We are looking for a passionate developer to work with the GitHub API to build a platform for measuring contributors performance/engagement across multiple open source projects over an extended time period.<br />
| Javascript, javascript framework, a sequel database, HTML, GitHub API <br />
| [https://mozillians.org/en-US/u/lucyeoh/ :Lucy]<br />
| [https://mozillians.org/en-US/u/bacharakis/ :Christos]<br />
| For questions or to express interest in this project please leave a comment in **[https://github.com/mozilla/Campus-Program/issues/363 this issue]** <br />
|-<br />
|}<br />
<br />
== Calendar ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== SeaMonkey ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Bugzilla ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Firefox Support (SUMO) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== QA ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Automation & Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! JS static analysis <br />
! Bring some static analysis in our Firefox Javascript code<br />
! Javascript experience, FLOW(?)<br />
! Sylvestre<br />
! same <br />
! This project aims to evaluate and integrate more static analysis for Javascript code into some key sections of the Firefox code. We will focus on deploying [https://github.com/facebook/flow FLOW] on some part of the code as a proof of concept. Depending on the results, this will be extended to more components and/or integration in the developer workflow.<br />
|-<br />
! C++ static analysis <br />
! Add new checkers specific to our base code<br />
! Strong C++ experience, clang<br />
! Sylvestre<br />
! Andi<br />
! In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns).<br />
|-<br />
! JSON in Sqlite<br />
! Query JSON Documents stored in Sqlite<br />
! Database, SQL, Python <br />
! Kyle Lahnakoski<br />
! Kyle Lahnakoski<br />
! [https://github.com/klahnakoski/JSONQueryExpressionTests/blob/master/docs/GSOC%20Proposal.md Details]<br />
|-<br />
! View details of performance test results<br />
! At the moment, Perfherder provides summarized views of results of Talos tests and others, but not the individual test results, athough the supporting data exists. This project will be about creating an easy-to-use web interface for visualizing this data that integrates well with the existing Perfherder views<br />
! HTML, CSS, JS, AngularJS<br />
! Will Lachance (:wlach)<br />
! Will Lachance (:wlach), Robert Wood (:rwood)<br />
! [https://wiki.mozilla.org/Auto-tools/Projects/Perfherder Perfherder wiki] / [https://bugzilla.mozilla.org/show_bug.cgi?id=1164891 bug] (don't be fooled by the fact that there is just one bug, there is easily enough work here to fill a gsoc project)<br />
|-<br />
! Regression localization from stack traces<br />
! Tool to blame a patch for causing a newly introduced crash.<br />
! C++, Python<br />
! Marco<br />
! Marco<br />
! The project would involve creating a tool that analyzes a set of stack traces, performing the intersection with the Firefox call graph (which can be generated using Clang) and a set of recently committed patches.<br />
|-<br />
! Improvements for crash clustering<br />
! Build a tool to improve crash clustering, currently based on the top method of the stack trace.<br />
! Python, C++<br />
! Marco<br />
! Marco<br />
! The project would involve finalizing https://github.com/marco-c/crashsimilarity; testing it more thoroughly; putting it in production.<br />
|}<br />
<br />
== Documentation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Mozilla Developer Network ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| MDN Documentation Tester WebExtension<br />
| Rewrite the [https://github.com/Elchi3/mdn-doc-tests MDN doc tester add-on] as a (well-tested, modern, documented) WebExtension that runs in Firefox, Chrome, Opera, and possibly Edge. Ideally reach feature parity with the existing Add-on SDK add-on and if time allows work on new test ideas that run against the MDN documentation. <br />
| Add-On/WebExtension experience (JavaScript, HTML, CSS)<br />
| Florian Scholz<br />
| Florian Scholz<br />
|}<br />
<br />
== Mozilla IT and Infrastructure ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Sync / Services ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Add-on SDK ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Foundation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Curriculum database for Learning site<br />
| Curriculum is currently accessed via the Teaching Activities page of the learning site (https://learning.mozilla.org/en-US/activities ). It's a pretty static experience. Users can browse through a list of modules, but there is no search capability. We have a prototype for a more interactive UX that allows users to search and filter curriculum modules and activities: https://flukeout.github.io/curriculum-search-react/. This project would require implementing the front-end, as well as wiring up the backend database (for which the model has been defined, but the data has not been migrated).<br />
| js, html, css, django, python<br />
| [https://mozillians.org/en-US/u/hannah/ hannah]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This would be a big win for the curriculum team and for the community members who make good use of our learning/teaching materials. The database could eventually be extended beyond the learning site and support other parts of the network as well.<br />
|}<br />
<br />
== OpenArt ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Release Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Improve Balrog's Admin API<br />
| Make Balrog's Admin API simpler and easier for clients to work with by moving to a [http://swagger.io/ Swagger]-based approach.<br />
| python,rest apis<br />
| Ben Hearsum<br />
| Ben Hearsum<br />
|}<br />
<br />
== Emscripten ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Rust ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! Rust + WebAssembly showcase<br />
! Rust and WebAssembly are going to be a great pair. Design and implement a simple and attractive web application, in Rust, that demonstrates the power of Rust on the web. Make fixes to upstream projects as necessary. Write a blog post about it.<br />
! Rust, web development<br />
! brson<br />
! brson, badboy<br />
! cc https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-emscripten/7627. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-1-md<br />
|-<br />
! Rust dashboard updates<br />
! Update rusty-dash.com to include additional metrics important to the project. This tool is vital to the day-to-day management of Rust, but it needs some dedicated attention to fulfill its promise.<br />
! Rust<br />
! brson<br />
! brson, aturon?, dikaiosune?<br />
! Follow on work to https://internals.rust-lang.org/t/the-rust-project-needs-much-better-visibility-into-important-metrics/3367<br />
|-<br />
! Rust reproducible builds<br />
! Make the Rust build process produce the identical binaries when run with identical configurations. This improves the security of the Rust ecosystem by allowing others to<br />
double-check the official Rust builds<br />
! Rust, compilers, systems programming<br />
! brson<br />
! brson, mw, Manishearth<br />
! https://internals.rust-lang.org/t/verifying-rustc-releases-with-reproducible-builds/4502. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-2-md<br />
|-<br />
! Abstract the Rust standard library<br />
! The Rust standard library is very portable, but can be very, very portable. Refactor the<br />
standard library to pull out a platform abstraction layer.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301<br />
|-<br />
! Rust cross-platform showcase project<br />
! Rust is very portable. Create a single showcase demo project that compiles for Windows, OS X, Linux, Android, iOS, wasm, and microcontrollers. Use real crates to accomplish some real task. Set up CI for all platforms on Travis. To be used as a teaching tool and for regression testing. Write a blog post.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! rustc micro-optimization bonanza<br />
! Just go hog wild finding microoptimizations in rustc. Write a blog post bragging about it.<br />
! Performance optimization<br />
! brson<br />
! brson, nnethercote?<br />
! <br />
|-<br />
! Rust-specific benchmark suite<br />
! Today Rust uses perf.rust-lang.org to track _compile time_ performance, but nothing to track _runtime_ performance. Work with the Rust developers to create a benchmark suite specific to Rust.<br />
! Programming<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! Rust crate semver compatibility tool<br />
! Rust libraries follow the semver spec for indicating API compatibility, but conformance to semver is not enforced in any way - it is up to crate authors to guarantee their crates can be upgraded correctly. With Rust's strong type system it should be possible to mechanically check whether crates<br />
are obeying semver. This would be a huge boon to the stability of the Rust ecosystem.<br />
! Static analysis<br />
! brson<br />
! brson, badboy<br />
! https://users.rust-lang.org/t/warnings-for-breaking-semver/1415 https://users.rust-lang.org/t/signature-based-api-comparison/2377 . Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-3-md<br />
|-<br />
! Reconstruct the Rust bootstrap chain<br />
! Rust is a self-hosted compiler, originally bootstrapped from OCaml, then self-bootstrapped several hundred times over the years. Today there is only one Rust 'lineage' of any note, the official compiler, but with compilers diversity of implementation is a security issue. If another party could replicate the<br />
Rust compiler it would provide assurance that the official isn't compromised by, or can recover from, a "trusting trust" attack. Although the historical information needed to rebootstrap Rust is mostly complete, it is not trivial to do. Construct a script that can reconstruct the modern Rust compiler from the original OCaml implementation, run it to completion, write a blog post.<br />
! Scripting, build systems<br />
! brson<br />
! acrichto<br />
! <br />
|-<br />
|}<br />
<br />
== Servo ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Off-main thread HTML parsing<br />
| [https://github.com/servo/servo/wiki/Off-main-thread-HTML-parsing-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| Anthony Ramine (nox)<br />
|<br />
|-<br />
| Extend ServiceWorker implementation<br />
| [https://github.com/servo/servo/wiki/More-ServiceWorker-support-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| <br />
|}<br />
<br />
== Security Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Tor Experiment<br />
| Create a an experimental extension that bundles and launches Tor. Benchmark and report the amount of data transferred, time-to-first-byte, bandwidth differentials.<br />
| Add-On creation experience, Programming in C/Javascript.<br />
| Tom Ritter<br />
| Tom Ritter<br />
|-<br />
| StopTrackware.org<br />
| Create a clearinghouse of trackware lists like StopBadware.org, by creating a Track-the-trackers add-on that uses data [http://www2016.net/proceedings/proceedings/p121.pdf "safeness" approach like Cliqz]<br />
| Add-On creation experience (JavaScript, HTML, CSS, Networking), Cloud Service experience (python or node.js)<br />
| Luke Crouch<br />
| Luke Crouch<br />
|<br />
|}<br />
<br />
== Localization ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Design screenshot-based localization interface in [https://pontoon.mozilla.org/ Pontoon]<br />
| We'd like to explore the feasibility of screenshot-based localization process in [https://pontoon.mozilla.org/ Pontoon] - Mozilla's translation tool. To provide more translation context, each string will be accompanied by a screenshot showing how the string is used in the application. Additionally, localizers will have the ability to navigate and filter strings by screenshots, as well as preview their translations in the localized screenshots.<br />
| Web standards and Design. Django is a plus.<br />
| [[User:Mathjazz|:mathjazz]]<br />
| [[User:Mathjazz|:mathjazz]]<br />
| <br />
|}<br />
<br />
== Build system ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Security Assurance ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|<br />
TLS 1.3 scanning in Mozilla TLS Observatory<br />
|<br />
[https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to improve the service to support scanning TLS 1.3 enabled endpoints, either by improving the [https://github.com/mozilla/cipherscan existing scanner] or writing an entirely new scanner.<br />
|<br />
Programming skills in C, Bash and Go. Strong understanding of TLS and micro-services architectures.<br />
|<br />
Julien Vehent<br />
|<br />
Julien Vehent<br />
|<br />
None<br />
|}<br />
<br />
== WADI ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Mozilla Science Lab ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| science.mozilla.org phase 2<br />
| Start phase 2 of the Mozilla Science Lab website by implementing<br />
* [https://github.com/mozilla/science.mozilla.org/issues/135 github authentication], <br />
* [https://github.com/mozilla/science.mozilla.org/issues/311 members homepage],<br />
* [https://github.com/mozilla/science.mozilla.org/issues/536 improve blog post readability], and add the<br />
* [https://github.com/mozilla/science.mozilla.org/issues/518 ability to join projects]<br />
| js, html, css, react, django, python<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/alanmoo/ alanmoo]<br />
| Detailed list and breakdown of all phase 2 work (only the tasks mentioned in details are expected to be completed by the student):<br />
* Phase 2.1 https://github.com/mozilla/science.mozilla.org/milestone/11<br />
* Phase 2.2 https://github.com/mozilla/science.mozilla.org/milestone/12 <br />
|-<br />
| Paper Badger - badgr migration & publisher integration<br />
| * Issuing badges to credit authors for their work on academic papers.<br />
* In the final steps of our badgekit to Badgr migration, add tests to make sure we have feature parity and make the switch to Badgr in production.<br />
* Implement a [https://github.com/mozillascience/PaperBadger/issues/160 journal submission system integration] with paper badger. This way, journals won't have to manually enter new badges in our system.<br />
| js, Badgr, APIs, authentication, node.js, react<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| github: https://github.com/mozillascience/paperbadger<br />
|-<br />
| Study Group participation visualization<br />
| [https://github.com/mozillascience/studyGroup study groups] meet all over the world, running regular events share skills, co-work and create community around open science. Right now there's no easy way to see all the study group events.<br />
* Write a script to collect data for all study group events happening world wide. Each study group is a [https://github.com/mozillascience/studyGroup fork of this template repo]. Each event is an issue in the repo ([https://github.com/UofTCoders/Events/issues example of events as issues]). [https://github.com/mozillascience/studyGroup/blob/gh-pages/scripts/updateCalendar.py here is a script] that collects all events for a specific repo. Please update this to collect data from all forks.<br />
* Visualize this data in a way that shows study group activity all around the world! viz: calendar, map<br />
| js, html, css, python (or another scripting language), data visualization<br />
| [https://mozillians.org/en-US/u/zannah/ Zannah]<br />
| [https://mozillians.org/en-US/u/aurelia/ Aurelia]<br />
| main study group repo: https://github.com/mozillascience/studyGroup<br />
|}<br />
<br />
<br />
== MozVR ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
== Connected Devices ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
<br />
<br />
== Thimble ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Address stability issues within Thimble<br />
| Main task: Get Thimble to write directly to AWS instead of writing to a database first.<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| Full list of issues affecting thimble's stability: https://github.com/mozilla/thimble.mozilla.org/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20bug%22%20-label%3A%22assigned%20to%20contributor%22<br />
|-<br />
| Integrating VR capabilities into Thimble<br />
| Goal: Thimble makes it easy to create multi user VR experiences, allowing the whole classroom to take virtual field trips through student projects.<br />
This is made possible by doing any of these:<br />
* Integration of the A-frame visual inspector, which gives the user lots of control over an object properties in a custom UI<br />
* Custom Quick-Edit UI for different types of objects<br />
* Develop large, visual menu of 3D objects make it easy to build interesting scenes quickly<br />
* A wide range of starter projects<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This work would be done in parallel with work done by [https://cdot.senecacollege.ca/ CDOT] on making Thimble a VR experience building tool<br />
thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|-<br />
| Thimble and Remote Mentorship<br />
| Goal: Use together.js to enable a multi-user environment on Thimble that is tailored to work as a virtual teaching environment for HTML/CSS/JS<br />
Details:<br />
* Fixing issues in together.js to make it functional<br />
* working on ways to sync state between Thimble's editor and the remote Thimble editor(s<br />
* Syncing filesystems across Thimble editors with WebRTC<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode17:Brainstorming&diff=1164356Community:SummerOfCode17:Brainstorming2017-03-02T18:39:35Z<p>Wlach: fix formatting</p>
<hr />
<div>Mozilla community members - submit proposals here for 2017 Google Summer of Code projects with Mozilla. (If this page looks empty, it's because accepted ideas have already been transferred to the [[Community:SummerOfCode17|official list]].) '''The''' absolute last '''deadline for submitting ideas''' in time to help us get accepted by Google '''is February 9th'''.<br />
<br />
<b>Are you a student looking to apply to SoC with Mozilla?</b> Your first stop should be the [[Community:SummerOfCode17|official list of ideas]]. This page is full of weird and whacky ideas, some of which are still on here for a reason - it could be that they are not properly defined, the wrong size, or don't have a mentor. That makes them less likely to get accepted. You <i>can</i>, of course, also submit your own ideas - you don't have to put an idea on this page and get it 'made official' in order to send in a proposal for it.<br />
<br />
==How To Write A Good Project Proposal==<br />
<br />
Before adding an proposal to this list, please consider the following:<br />
<br />
* '''Be specific'''. It's hard to understand the impact of, or the size of, vague proposals.<br />
* '''Consider size'''. The student has eight weeks to design, code, test and document the proposal. It needs to fill, but not overfill, that time.<br />
* '''Do your research'''. Support the idea with well-researched links.<br />
* '''Don't morph other people's ideas'''. If you have a related idea, place it next to the existing one, or add a comment. <br />
* '''Insert only your own name into the Mentor column''', and then only if you are willing to take on the responsibility. If you think the SoC admins won't know who you are, leave contact details.<br />
* '''Check back regularly'''. The administrators may have questions about your idea that you will need to answer.<br />
* '''Know when to give up'''. If you've added the same idea for the last three years and it hasn't made it to the official page, perhaps you can predict what will happen this time.<br />
<br />
==Suggestion List==<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
== Mozilla Platform (Gecko) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| TaskCluster JSON Parameterization<br />
| Build a powerful JSON parameterization system and use it everywhere<br />
| JavaScript, Python<br />
| :dustin<br />
| :dustin<br />
| The json-e language supports complex transformations of JSON data. The project involves completing the specification, implementation (in two languages), and documentation of this language, then using it to support Gecko action and decision task and users of taskcluster-github and taskcluster-hooks. Success here means that the language is complete and in active use in at least one of the listed contexts.<br />
|-<br />
| Livelog Proxy <br />
| Write a server that privileged-clients can open a HTTPS connection to in-order to expose a webhook that http-clients can call.<br />
When normal http-clients access the exposed webhooks the connection will be reverse proxied to the<br />
privileged-clients over their out-going connection.<br />
| golang, github, http, web sockets, node.js<br />
| :jonasfj<br />
| :jonasfj<br />
| This is like to ngrok and localtunnel.me, read up on those. For performance reasons server should be written in golang, with client libraries in golang and node.js.<br />
| <br />
|}<br />
<br />
== Firefox ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| about:telemetry redesign<br />
| about:telemetry is present on all builds of Firefox as a way for users to view the data being stored and sent via Telemetry. It was built before Firefox had multi-process Telemetry and without a clear design. This has resulted in a confusing HTML UI and barely-comprehensible JS.<br />
| webtech (HTML+CSS+JS) and Design<br />
| :chutten<br />
| :chutten<br />
| <br />
|-<br />
| Boost Session Restore performance<br />
| Session (Re)store is important as a key feature of Firefox. Many people rely on it to re-open a tab from the past or recover from an unfortunate power outage. But making it blazing fast has not been our primary focus, until now. Your goal will be to help us make restoring any session snappy and blazing fast. Expect to learn a lot about the Firefox internals and interact with many core engineers during the SoC course.<br />
| JavaScript<br />
| :mikedeboer<br />
| :mikedeboer, :dao<br />
|<br />
|}<br />
<br />
== Firefox Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Developer Tools for Designers<br />
| In Firefox Developer Tools, we're aiming to implement visual tools that will help developers and designers edit CSS properties visually. Since CSS is very declarative by nature, we can provide tools that are commonly found in designer tools in order to bridge the learning gap of new developers and designers learning CSS and allow them to edit complex CSS features such as CSS filters, blend, clip-path, layout, etc without knowing what the properties are. Going beyond editing CSS, we also need to think about how we can visually inspect the CSS style transformations that can impact the layout. Finally, we are looking for ways to better export and see the changes made in the Inspector tool so that we can take the work we have done in the Developer Tools and apply that to your local source files.<br />
| JavaScript, HTML, CSS, React, Redux<br />
| [[User:Gl|:gl]]<br />
| [[User:Gl|:gl]]<br />
| Starting meta bug for inspiration [https://bugzilla.mozilla.org/show_bug.cgi?id=1258390 Bug 1258390]<br />
|}<br />
<br />
== Firefox for Android ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Implement WebExtension APIs<br />
| [https://developer.mozilla.org/en-US/Add-ons/WebExtensions WebExtensions] are a cross-browser system for developing browser add-ons. [http://arewewebextensionsyet.com/ Not all APIs are supported] by Firefox for Android yet.<br />
The goal of this project is to:<br />
* Implement some of the missing APIs ([https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus contextMenus], [https://developer.chrome.com/extensions/browsingData browsingData], [identity?], [proxy?])<br />
* Add automated test for those APIs<br />
* Improve the tooling for building WebExtensions for Firefox for Android.<br />
| Java, JavaScript, Android<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
|}<br />
<br />
== Thunderbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Instantbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
| Matrix protocol support<br />
| [http://matrix.org/ Matrix] is an open, decentralized protocol for instant messaging (and more!) It has bridges to many other networks and protocol, e.g. IRC, Slack, and more. Initial support for Matrix was added in {{bug|1199855}}, but there's a lot to do still:<br />
<br />
* Support more features from the Matrix SDK (room topics, typing notifications, read receipts, etc.)<br />
* Support one-on-one conversations.<br />
* Add tests specific to Matrix.<br />
* Improve the Matrix JS-SDK that Instantbird and Thunderbird depend on.<br />
* Improving and expanding shared code and APIs used by all JavaScript protocol plugins (IRC, XMPP, Yahoo and Twitter).<br />
* Improving documentation of the process for adding a protocol to Instantbird/Thunderbird.<br />
* Using the Matrix protocol on a day-to-day basis to dog-food the code and see what else is missing.<br />
| JavaScript, XPCOM, experience using HTTP APIs<br />
| Patrick Cloke [:clokep]<br />
| [mailto:clokep@gmail.com clokep]<br />
| If you're interested in supporting a different protocol, please come talk to us. Protocols we're interested in: Matrix, Google Hangouts, Facebook, Bonjour, TextSecure/Signal, Telegram, or others!<br />
|}<br />
<br />
== Calendar ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== SeaMonkey ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Bugzilla ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Firefox Support (SUMO) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== QA ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Automation & Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! JS static analysis <br />
! Bring some static analysis in our Firefox Javascript code<br />
! Javascript experience, FLOW(?)<br />
! Sylvestre<br />
! same <br />
! This project aims to evaluate and integrate more static analysis for Javascript code into some key sections of the Firefox code. We will focus on deploying [https://github.com/facebook/flow FLOW] on some part of the code as a proof of concept. Depending on the results, this will be extended to more components and/or integration in the developer workflow.<br />
|-<br />
! C++ static analysis <br />
! Add new checkers specific to our base code<br />
! Strong C++ experience, clang<br />
! Sylvestre<br />
! Andi<br />
! In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns).<br />
|-<br />
! JSON in Sqlite<br />
! Query JSON Documents stored in Sqlite<br />
! Database, SQL, Python <br />
! Kyle Lahnakoski<br />
! Kyle Lahnakoski<br />
! [https://github.com/klahnakoski/JSONQueryExpressionTests/blob/master/docs/GSOC%20Proposal.md Details]<br />
|-<br />
! View details of performance test results<br />
! At the moment, Perfherder provides summarized views of results of Talos tests and others, but not the individual test results, athough the supporting data exists. This project will be about creating an easy-to-use web interface for visualizing this data that integrates well with the existing Perfherder views<br />
! HTML, CSS, JS, AngularJS<br />
! Will Lachance (:wlach)<br />
! Will Lachance (:wlach), Robert Wood (:rwood)<br />
! [https://wiki.mozilla.org/Auto-tools/Projects/Perfherder Perfherder wiki] / [https://bugzilla.mozilla.org/show_bug.cgi?id=1164891 bug] (don't be fooled by the fact that there is just one bug, there is easily enough work here to fill a gsoc project)<br />
|-<br />
! Regression localization from stack traces<br />
! Tool to blame a patch for causing a newly introduced crash.<br />
! C++, Python<br />
! Marco<br />
! Marco<br />
! The project would involve creating a tool that analyzes a set of stack traces, performing the intersection with the Firefox call graph (which can be generated using Clang) and a set of recently committed patches.<br />
|-<br />
! Improvements for crash clustering<br />
! Build a tool to improve crash clustering, currently based on the top method of the stack trace.<br />
! Python, C++<br />
! Marco<br />
! Marco<br />
! The project would involve finalizing https://github.com/marco-c/crashsimilarity; testing it more thoroughly; putting it in production.<br />
|-<br />
! GraphQL interface to Treeherder<br />
! This project would be about creating a GraphQL interface to Treeherder's datastore, as a complement to the existing REST API <br />
! Django, MySQL, Graphene<br />
! Will Lachance (:wlach)<br />
! Will Lachance (:wlach)<br />
! [https://wiki.mozilla.org/Auto-tools/Projects/Treeherder Treeherder wiki] / [https://bugzilla.mozilla.org/show_bug.cgi?id=1332457 bug] (don't be fooled by the fact that there is just one bug, there is easily enough work here to fill a gsoc project)<br />
|}<br />
<br />
== Documentation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Mozilla Developer Network ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| MDN Documentation Tester WebExtension<br />
| Rewrite the [https://github.com/Elchi3/mdn-doc-tests MDN doc tester add-on] as a (well-tested, modern, documented) WebExtension that runs in Firefox, Chrome, Opera, and possibly Edge. Ideally reach feature parity with the existing Add-on SDK add-on and if time allows work on new test ideas that run against the MDN documentation. <br />
| Add-On/WebExtension experience (JavaScript, HTML, CSS)<br />
| Florian Scholz<br />
| Florian Scholz<br />
|}<br />
<br />
== Mozilla IT and Infrastructure ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Sync / Services ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Add-on SDK ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Foundation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Curriculum database for Learning site<br />
| Curriculum is currently accessed via the Teaching Activities page of the learning site (https://learning.mozilla.org/en-US/activities ). It's a pretty static experience. Users can browse through a list of modules, but there is no search capability. We have a prototype for a more interactive UX that allows users to search and filter curriculum modules and activities: https://flukeout.github.io/curriculum-search-react/. This project would require implementing the front-end, as well as wiring up the backend database (for which the model has been defined, but the data has not been migrated).<br />
| js, html, css, django, python<br />
| [https://mozillians.org/en-US/u/hannah/ hannah]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This would be a big win for the curriculum team and for the community members who make good use of our learning/teaching materials. The database could eventually be extended beyond the learning site and support other parts of the network as well.<br />
|}<br />
<br />
== OpenArt ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Release Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Improve Balrog's Admin API<br />
| Make Balrog's Admin API simpler and easier for clients to work with by moving to a [http://swagger.io/ Swagger]-based approach.<br />
| python,rest apis<br />
| Ben Hearsum<br />
| Ben Hearsum<br />
|}<br />
<br />
== Emscripten ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Rust ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! Rust + WebAssembly showcase<br />
! Rust and WebAssembly are going to be a great pair. Design and implement a simple and attractive web application, in Rust, that demonstrates the power of Rust on the web. Make fixes to upstream projects as necessary. Write a blog post about it.<br />
! Rust, web development<br />
! brson<br />
! brson, badboy<br />
! cc https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-emscripten/7627. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-1-md<br />
|-<br />
! Rust dashboard updates<br />
! Update rusty-dash.com to include additional metrics important to the project. This tool is vital to the day-to-day management of Rust, but it needs some dedicated attention to fulfill its promise.<br />
! Rust<br />
! brson<br />
! brson, aturon?, dikaiosune?<br />
! Follow on work to https://internals.rust-lang.org/t/the-rust-project-needs-much-better-visibility-into-important-metrics/3367<br />
|-<br />
! Rust reproducible builds<br />
! Make the Rust build process produce the identical binaries when run with identical configurations. This improves the security of the Rust ecosystem by allowing others to<br />
double-check the official Rust builds<br />
! Rust, compilers, systems programming<br />
! brson<br />
! brson, mw, Manishearth<br />
! https://internals.rust-lang.org/t/verifying-rustc-releases-with-reproducible-builds/4502. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-2-md<br />
|-<br />
! Abstract the Rust standard library<br />
! The Rust standard library is very portable, but can be very, very portable. Refactor the<br />
standard library to pull out a platform abstraction layer.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301<br />
|-<br />
! Rust cross-platform showcase project<br />
! Rust is very portable. Create a single showcase demo project that compiles for Windows, OS X, Linux, Android, iOS, wasm, and microcontrollers. Use real crates to accomplish some real task. Set up CI for all platforms on Travis. To be used as a teaching tool and for regression testing. Write a blog post.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! rustc micro-optimization bonanza<br />
! Just go hog wild finding microoptimizations in rustc. Write a blog post bragging about it.<br />
! Performance optimization<br />
! brson<br />
! brson, nnethercote?<br />
! <br />
|-<br />
! Rust-specific benchmark suite<br />
! Today Rust uses perf.rust-lang.org to track _compile time_ performance, but nothing to track _runtime_ performance. Work with the Rust developers to create a benchmark suite specific to Rust.<br />
! Programming<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! Rust crate semver compatibility tool<br />
! Rust libraries follow the semver spec for indicating API compatibility, but conformance to semver is not enforced in any way - it is up to crate authors to guarantee their crates can be upgraded correctly. With Rust's strong type system it should be possible to mechanically check whether crates<br />
are obeying semver. This would be a huge boon to the stability of the Rust ecosystem.<br />
! Static analysis<br />
! brson<br />
! brson, badboy<br />
! https://users.rust-lang.org/t/warnings-for-breaking-semver/1415 https://users.rust-lang.org/t/signature-based-api-comparison/2377 . Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-3-md<br />
|-<br />
! Reconstruct the Rust bootstrap chain<br />
! Rust is a self-hosted compiler, originally bootstrapped from OCaml, then self-bootstrapped several hundred times over the years. Today there is only one Rust 'lineage' of any note, the official compiler, but with compilers diversity of implementation is a security issue. If another party could replicate the<br />
Rust compiler it would provide assurance that the official isn't compromised by, or can recover from, a "trusting trust" attack. Although the historical information needed to rebootstrap Rust is mostly complete, it is not trivial to do. Construct a script that can reconstruct the modern Rust compiler from the original OCaml implementation, run it to completion, write a blog post.<br />
! Scripting, build systems<br />
! brson<br />
! acrichto<br />
! <br />
|-<br />
|}<br />
<br />
== Servo ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Off-main thread HTML parsing<br />
| [https://github.com/servo/servo/wiki/Off-main-thread-HTML-parsing-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| Anthony Ramine (nox)<br />
|<br />
|-<br />
| Extend ServiceWorker implementation<br />
| [https://github.com/servo/servo/wiki/More-ServiceWorker-support-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| <br />
|}<br />
<br />
== Security Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Tor Experiment<br />
| Create a an experimental extension that bundles and launches Tor. Benchmark and report the amount of data transferred, time-to-first-byte, bandwidth differentials.<br />
| Add-On creation experience, Programming in C/Javascript.<br />
| Tom Ritter<br />
| Tom Ritter<br />
|-<br />
| StopTrackware.org<br />
| Create a clearinghouse of trackware lists like StopBadware.org, by creating a Track-the-trackers add-on that uses data [http://www2016.net/proceedings/proceedings/p121.pdf "safeness" approach like Cliqz]<br />
| Add-On creation experience (JavaScript, HTML, CSS, Networking), Cloud Service experience (python or node.js)<br />
| Luke Crouch<br />
| Luke Crouch<br />
|<br />
|}<br />
<br />
== Localization ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Design screenshot-based localization interface in [https://pontoon.mozilla.org/ Pontoon]<br />
| We'd like to explore the feasibility of screenshot-based localization process in [https://pontoon.mozilla.org/ Pontoon] - Mozilla's translation tool. To provide more translation context, each string will be accompanied by a screenshot showing how the string is used in the application. Additionally, localizers will have the ability to navigate and filter strings by screenshots, as well as preview their translations in the localized screenshots.<br />
| Web standards and Design. Django is a plus.<br />
| [[User:Mathjazz|:mathjazz]]<br />
| [[User:Mathjazz|:mathjazz]]<br />
| <br />
|}<br />
<br />
== Build system ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Security Assurance ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|<br />
TLS 1.3 scanning in Mozilla TLS Observatory<br />
|<br />
[https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to improve the service to support scanning TLS 1.3 enabled endpoints, either by improving the [https://github.com/mozilla/cipherscan existing scanner] or writing an entirely new scanner.<br />
|<br />
Programming skills in C, Bash and Go. Strong understanding of TLS and micro-services architectures.<br />
|<br />
Julien Vehent<br />
|<br />
Julien Vehent<br />
|<br />
None<br />
|}<br />
<br />
== WADI ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Mozilla Science Lab ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| science.mozilla.org phase 2<br />
| Start phase 2 of the Mozilla Science Lab website by implementing<br />
* [https://github.com/mozilla/science.mozilla.org/issues/135 github authentication], <br />
* [https://github.com/mozilla/science.mozilla.org/issues/311 members homepage],<br />
* [https://github.com/mozilla/science.mozilla.org/issues/536 improve blog post readability], and add the<br />
* [https://github.com/mozilla/science.mozilla.org/issues/518 ability to join projects]<br />
| js, html, css, react, django, python<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/alanmoo/ alanmoo]<br />
| Detailed list and breakdown of all phase 2 work (only the tasks mentioned in details are expected to be completed by the student):<br />
* Phase 2.1 https://github.com/mozilla/science.mozilla.org/milestone/11<br />
* Phase 2.2 https://github.com/mozilla/science.mozilla.org/milestone/12 <br />
|-<br />
| Paper Badger - badgr migration & publisher integration<br />
| * Issuing badges to credit authors for their work on academic papers.<br />
* In the final steps of our badgekit to Badgr migration, add tests to make sure we have feature parity and make the switch to Badgr in production.<br />
* Implement a [https://github.com/mozillascience/PaperBadger/issues/160 journal submission system integration] with paper badger. This way, journals won't have to manually enter new badges in our system.<br />
| js, Badgr, APIs, authentication, node.js, react<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| github: https://github.com/mozillascience/paperbadger<br />
|-<br />
| Study Group participation visualization<br />
| [https://github.com/mozillascience/studyGroup study groups] meet all over the world, running regular events share skills, co-work and create community around open science. Right now there's no easy way to see all the study group events.<br />
* Write a script to collect data for all study group events happening world wide. Each study group is a [https://github.com/mozillascience/studyGroup fork of this template repo]. Each event is an issue in the repo ([https://github.com/UofTCoders/Events/issues example of events as issues]). [https://github.com/mozillascience/studyGroup/blob/gh-pages/scripts/updateCalendar.py here is a script] that collects all events for a specific repo. Please update this to collect data from all forks.<br />
* Visualize this data in a way that shows study group activity all around the world! viz: calendar, map<br />
| js, html, css, python (or another scripting language), data visualization<br />
| [https://mozillians.org/en-US/u/zannah/ Zannah]<br />
| [https://mozillians.org/en-US/u/aurelia/ Aurelia]<br />
| main study group repo: https://github.com/mozillascience/studyGroup<br />
|}<br />
<br />
<br />
== MozVR ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
== Connected Devices ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
<br />
<br />
== Thimble ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Address stability issues within Thimble<br />
| Main task: Get Thimble to write directly to AWS instead of writing to a database first.<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| Full list of issues affecting thimble's stability: https://github.com/mozilla/thimble.mozilla.org/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20bug%22%20-label%3A%22assigned%20to%20contributor%22<br />
|-<br />
| Integrating VR capabilities into Thimble<br />
| Goal: Thimble makes it easy to create multi user VR experiences, allowing the whole classroom to take virtual field trips through student projects.<br />
This is made possible by doing any of these:<br />
* Integration of the A-frame visual inspector, which gives the user lots of control over an object properties in a custom UI<br />
* Custom Quick-Edit UI for different types of objects<br />
* Develop large, visual menu of 3D objects make it easy to build interesting scenes quickly<br />
* A wide range of starter projects<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This work would be done in parallel with work done by [https://cdot.senecacollege.ca/ CDOT] on making Thimble a VR experience building tool<br />
thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|-<br />
| Thimble and Remote Mentorship<br />
| Goal: Use together.js to enable a multi-user environment on Thimble that is tailored to work as a virtual teaching environment for HTML/CSS/JS<br />
Details:<br />
* Fixing issues in together.js to make it functional<br />
* working on ways to sync state between Thimble's editor and the remote Thimble editor(s<br />
* Syncing filesystems across Thimble editors with WebRTC<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=Community:SummerOfCode17:Brainstorming&diff=1164355Community:SummerOfCode17:Brainstorming2017-03-02T18:37:08Z<p>Wlach: /* Automation & Tools */ graphql</p>
<hr />
<div>Mozilla community members - submit proposals here for 2017 Google Summer of Code projects with Mozilla. (If this page looks empty, it's because accepted ideas have already been transferred to the [[Community:SummerOfCode17|official list]].) '''The''' absolute last '''deadline for submitting ideas''' in time to help us get accepted by Google '''is February 9th'''.<br />
<br />
<b>Are you a student looking to apply to SoC with Mozilla?</b> Your first stop should be the [[Community:SummerOfCode17|official list of ideas]]. This page is full of weird and whacky ideas, some of which are still on here for a reason - it could be that they are not properly defined, the wrong size, or don't have a mentor. That makes them less likely to get accepted. You <i>can</i>, of course, also submit your own ideas - you don't have to put an idea on this page and get it 'made official' in order to send in a proposal for it.<br />
<br />
==How To Write A Good Project Proposal==<br />
<br />
Before adding an proposal to this list, please consider the following:<br />
<br />
* '''Be specific'''. It's hard to understand the impact of, or the size of, vague proposals.<br />
* '''Consider size'''. The student has eight weeks to design, code, test and document the proposal. It needs to fill, but not overfill, that time.<br />
* '''Do your research'''. Support the idea with well-researched links.<br />
* '''Don't morph other people's ideas'''. If you have a related idea, place it next to the existing one, or add a comment. <br />
* '''Insert only your own name into the Mentor column''', and then only if you are willing to take on the responsibility. If you think the SoC admins won't know who you are, leave contact details.<br />
* '''Check back regularly'''. The administrators may have questions about your idea that you will need to answer.<br />
* '''Know when to give up'''. If you've added the same idea for the last three years and it hasn't made it to the official page, perhaps you can predict what will happen this time.<br />
<br />
==Suggestion List==<br />
<br />
[[SummerOfCode|Here are the ideas lists from previous years]].<br />
<br />
Proposals can be in almost any part of the Mozilla project - don't be fooled by the "Code" in "Summer of Code". If there is no category below for your part of Mozilla, add one!<br />
<br />
== Mozilla Platform (Gecko) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| TaskCluster JSON Parameterization<br />
| Build a powerful JSON parameterization system and use it everywhere<br />
| JavaScript, Python<br />
| :dustin<br />
| :dustin<br />
| The json-e language supports complex transformations of JSON data. The project involves completing the specification, implementation (in two languages), and documentation of this language, then using it to support Gecko action and decision task and users of taskcluster-github and taskcluster-hooks. Success here means that the language is complete and in active use in at least one of the listed contexts.<br />
|-<br />
| Livelog Proxy <br />
| Write a server that privileged-clients can open a HTTPS connection to in-order to expose a webhook that http-clients can call.<br />
When normal http-clients access the exposed webhooks the connection will be reverse proxied to the<br />
privileged-clients over their out-going connection.<br />
| golang, github, http, web sockets, node.js<br />
| :jonasfj<br />
| :jonasfj<br />
| This is like to ngrok and localtunnel.me, read up on those. For performance reasons server should be written in golang, with client libraries in golang and node.js.<br />
| <br />
|}<br />
<br />
== Firefox ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| about:telemetry redesign<br />
| about:telemetry is present on all builds of Firefox as a way for users to view the data being stored and sent via Telemetry. It was built before Firefox had multi-process Telemetry and without a clear design. This has resulted in a confusing HTML UI and barely-comprehensible JS.<br />
| webtech (HTML+CSS+JS) and Design<br />
| :chutten<br />
| :chutten<br />
| <br />
|-<br />
| Boost Session Restore performance<br />
| Session (Re)store is important as a key feature of Firefox. Many people rely on it to re-open a tab from the past or recover from an unfortunate power outage. But making it blazing fast has not been our primary focus, until now. Your goal will be to help us make restoring any session snappy and blazing fast. Expect to learn a lot about the Firefox internals and interact with many core engineers during the SoC course.<br />
| JavaScript<br />
| :mikedeboer<br />
| :mikedeboer, :dao<br />
|<br />
|}<br />
<br />
== Firefox Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Developer Tools for Designers<br />
| In Firefox Developer Tools, we're aiming to implement visual tools that will help developers and designers edit CSS properties visually. Since CSS is very declarative by nature, we can provide tools that are commonly found in designer tools in order to bridge the learning gap of new developers and designers learning CSS and allow them to edit complex CSS features such as CSS filters, blend, clip-path, layout, etc without knowing what the properties are. Going beyond editing CSS, we also need to think about how we can visually inspect the CSS style transformations that can impact the layout. Finally, we are looking for ways to better export and see the changes made in the Inspector tool so that we can take the work we have done in the Developer Tools and apply that to your local source files.<br />
| JavaScript, HTML, CSS, React, Redux<br />
| [[User:Gl|:gl]]<br />
| [[User:Gl|:gl]]<br />
| Starting meta bug for inspiration [https://bugzilla.mozilla.org/show_bug.cgi?id=1258390 Bug 1258390]<br />
|}<br />
<br />
== Firefox for Android ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Implement WebExtension APIs<br />
| [https://developer.mozilla.org/en-US/Add-ons/WebExtensions WebExtensions] are a cross-browser system for developing browser add-ons. [http://arewewebextensionsyet.com/ Not all APIs are supported] by Firefox for Android yet.<br />
The goal of this project is to:<br />
* Implement some of the missing APIs ([https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/contextMenus contextMenus], [https://developer.chrome.com/extensions/browsingData browsingData], [identity?], [proxy?])<br />
* Add automated test for those APIs<br />
* Improve the tooling for building WebExtensions for Firefox for Android.<br />
| Java, JavaScript, Android<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
| [https://mozillians.org/en-US/u/sebastian.kaspari/ :sebastian]<br />
|}<br />
<br />
== Thunderbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Instantbird ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
| Matrix protocol support<br />
| [http://matrix.org/ Matrix] is an open, decentralized protocol for instant messaging (and more!) It has bridges to many other networks and protocol, e.g. IRC, Slack, and more. Initial support for Matrix was added in {{bug|1199855}}, but there's a lot to do still:<br />
<br />
* Support more features from the Matrix SDK (room topics, typing notifications, read receipts, etc.)<br />
* Support one-on-one conversations.<br />
* Add tests specific to Matrix.<br />
* Improve the Matrix JS-SDK that Instantbird and Thunderbird depend on.<br />
* Improving and expanding shared code and APIs used by all JavaScript protocol plugins (IRC, XMPP, Yahoo and Twitter).<br />
* Improving documentation of the process for adding a protocol to Instantbird/Thunderbird.<br />
* Using the Matrix protocol on a day-to-day basis to dog-food the code and see what else is missing.<br />
| JavaScript, XPCOM, experience using HTTP APIs<br />
| Patrick Cloke [:clokep]<br />
| [mailto:clokep@gmail.com clokep]<br />
| If you're interested in supporting a different protocol, please come talk to us. Protocols we're interested in: Matrix, Google Hangouts, Facebook, Bonjour, TextSecure/Signal, Telegram, or others!<br />
|}<br />
<br />
== Calendar ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== SeaMonkey ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Bugzilla ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Firefox Support (SUMO) ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== QA ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Automation & Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! JS static analysis <br />
! Bring some static analysis in our Firefox Javascript code<br />
! Javascript experience, FLOW(?)<br />
! Sylvestre<br />
! same <br />
! This project aims to evaluate and integrate more static analysis for Javascript code into some key sections of the Firefox code. We will focus on deploying [https://github.com/facebook/flow FLOW] on some part of the code as a proof of concept. Depending on the results, this will be extended to more components and/or integration in the developer workflow.<br />
|-<br />
! C++ static analysis <br />
! Add new checkers specific to our base code<br />
! Strong C++ experience, clang<br />
! Sylvestre<br />
! Andi<br />
! In order to tackle issues during the development phase, Mozilla wrote a [https://hg.mozilla.org/mozilla-central/file/tip/build/clang-plugin bunch of static analyzers checkers] based on [http://clang.llvm.org/extra/clang-tidy/ clang-tidy]. In this project, we will focus on writing more checkers (either generic to C/C++ or specific to Gecko programming patterns).<br />
|-<br />
! JSON in Sqlite<br />
! Query JSON Documents stored in Sqlite<br />
! Database, SQL, Python <br />
! Kyle Lahnakoski<br />
! Kyle Lahnakoski<br />
! [https://github.com/klahnakoski/JSONQueryExpressionTests/blob/master/docs/GSOC%20Proposal.md Details]<br />
|-<br />
! View details of performance test results<br />
! At the moment, Perfherder provides summarized views of results of Talos tests and others, but not the individual test results, athough the supporting data exists. This project will be about creating an easy-to-use web interface for visualizing this data that integrates well with the existing Perfherder views<br />
! HTML, CSS, JS, AngularJS<br />
! Will Lachance (:wlach)<br />
! Will Lachance (:wlach), Robert Wood (:rwood)<br />
! [https://wiki.mozilla.org/Auto-tools/Projects/Perfherder Perfherder wiki] / [https://bugzilla.mozilla.org/show_bug.cgi?id=1164891 bug] (don't be fooled by the fact that there is just one bug, there is easily enough work here to fill a gsoc project)<br />
|-<br />
! Regression localization from stack traces<br />
! Tool to blame a patch for causing a newly introduced crash.<br />
! C++, Python<br />
! Marco<br />
! Marco<br />
! The project would involve creating a tool that analyzes a set of stack traces, performing the intersection with the Firefox call graph (which can be generated using Clang) and a set of recently committed patches.<br />
|-<br />
! Improvements for crash clustering<br />
! Build a tool to improve crash clustering, currently based on the top method of the stack trace.<br />
! Python, C++<br />
! Marco<br />
! Marco<br />
! The project would involve finalizing https://github.com/marco-c/crashsimilarity; testing it more thoroughly; putting it in production.<br />
|-<br />
! GraphQL interface to Treeherder<br />
! This project would be about creating a GraphQL interface to Treeherder's datastore, as a complement to the existing REST API ! Django, MySQL, Graphene<br />
! Will Lachance (:wlach)<br />
! Will Lachance (:wlach)<br />
! [https://wiki.mozilla.org/Auto-tools/Projects/Treeherder Treeherder wiki] / [https://bugzilla.mozilla.org/show_bug.cgi?id=1332457 bug] (don't be fooled by the fact that there is just one bug, there is easily enough work here to fill a gsoc project)|}<br />
<br />
== Documentation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Mozilla Developer Network ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| MDN Documentation Tester WebExtension<br />
| Rewrite the [https://github.com/Elchi3/mdn-doc-tests MDN doc tester add-on] as a (well-tested, modern, documented) WebExtension that runs in Firefox, Chrome, Opera, and possibly Edge. Ideally reach feature parity with the existing Add-on SDK add-on and if time allows work on new test ideas that run against the MDN documentation. <br />
| Add-On/WebExtension experience (JavaScript, HTML, CSS)<br />
| Florian Scholz<br />
| Florian Scholz<br />
|}<br />
<br />
== Mozilla IT and Infrastructure ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Sync / Services ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Developer Tools ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Add-on SDK ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Foundation ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Curriculum database for Learning site<br />
| Curriculum is currently accessed via the Teaching Activities page of the learning site (https://learning.mozilla.org/en-US/activities ). It's a pretty static experience. Users can browse through a list of modules, but there is no search capability. We have a prototype for a more interactive UX that allows users to search and filter curriculum modules and activities: https://flukeout.github.io/curriculum-search-react/. This project would require implementing the front-end, as well as wiring up the backend database (for which the model has been defined, but the data has not been migrated).<br />
| js, html, css, django, python<br />
| [https://mozillians.org/en-US/u/hannah/ hannah]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This would be a big win for the curriculum team and for the community members who make good use of our learning/teaching materials. The database could eventually be extended beyond the learning site and support other parts of the network as well.<br />
|}<br />
<br />
== OpenArt ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Release Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Improve Balrog's Admin API<br />
| Make Balrog's Admin API simpler and easier for clients to work with by moving to a [http://swagger.io/ Swagger]-based approach.<br />
| python,rest apis<br />
| Ben Hearsum<br />
| Ben Hearsum<br />
|}<br />
<br />
== Emscripten ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|}<br />
<br />
== Rust ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
! Rust + WebAssembly showcase<br />
! Rust and WebAssembly are going to be a great pair. Design and implement a simple and attractive web application, in Rust, that demonstrates the power of Rust on the web. Make fixes to upstream projects as necessary. Write a blog post about it.<br />
! Rust, web development<br />
! brson<br />
! brson, badboy<br />
! cc https://users.rust-lang.org/t/compiling-to-the-web-with-rust-and-emscripten/7627. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-1-md<br />
|-<br />
! Rust dashboard updates<br />
! Update rusty-dash.com to include additional metrics important to the project. This tool is vital to the day-to-day management of Rust, but it needs some dedicated attention to fulfill its promise.<br />
! Rust<br />
! brson<br />
! brson, aturon?, dikaiosune?<br />
! Follow on work to https://internals.rust-lang.org/t/the-rust-project-needs-much-better-visibility-into-important-metrics/3367<br />
|-<br />
! Rust reproducible builds<br />
! Make the Rust build process produce the identical binaries when run with identical configurations. This improves the security of the Rust ecosystem by allowing others to<br />
double-check the official Rust builds<br />
! Rust, compilers, systems programming<br />
! brson<br />
! brson, mw, Manishearth<br />
! https://internals.rust-lang.org/t/verifying-rustc-releases-with-reproducible-builds/4502. Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-2-md<br />
|-<br />
! Abstract the Rust standard library<br />
! The Rust standard library is very portable, but can be very, very portable. Refactor the<br />
standard library to pull out a platform abstraction layer.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! https://internals.rust-lang.org/t/refactoring-std-for-ultimate-portability/4301<br />
|-<br />
! Rust cross-platform showcase project<br />
! Rust is very portable. Create a single showcase demo project that compiles for Windows, OS X, Linux, Android, iOS, wasm, and microcontrollers. Use real crates to accomplish some real task. Set up CI for all platforms on Travis. To be used as a teaching tool and for regression testing. Write a blog post.<br />
! Moderate Rust experience<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! rustc micro-optimization bonanza<br />
! Just go hog wild finding microoptimizations in rustc. Write a blog post bragging about it.<br />
! Performance optimization<br />
! brson<br />
! brson, nnethercote?<br />
! <br />
|-<br />
! Rust-specific benchmark suite<br />
! Today Rust uses perf.rust-lang.org to track _compile time_ performance, but nothing to track _runtime_ performance. Work with the Rust developers to create a benchmark suite specific to Rust.<br />
! Programming<br />
! brson<br />
! brson<br />
! <br />
|-<br />
! Rust crate semver compatibility tool<br />
! Rust libraries follow the semver spec for indicating API compatibility, but conformance to semver is not enforced in any way - it is up to crate authors to guarantee their crates can be upgraded correctly. With Rust's strong type system it should be possible to mechanically check whether crates<br />
are obeying semver. This would be a huge boon to the stability of the Rust ecosystem.<br />
! Static analysis<br />
! brson<br />
! brson, badboy<br />
! https://users.rust-lang.org/t/warnings-for-breaking-semver/1415 https://users.rust-lang.org/t/signature-based-api-comparison/2377 . Full proposal: https://gist.github.com/brson/c936d0f5e5ec8c806a41e23e87455665#file-rust-proj-3-md<br />
|-<br />
! Reconstruct the Rust bootstrap chain<br />
! Rust is a self-hosted compiler, originally bootstrapped from OCaml, then self-bootstrapped several hundred times over the years. Today there is only one Rust 'lineage' of any note, the official compiler, but with compilers diversity of implementation is a security issue. If another party could replicate the<br />
Rust compiler it would provide assurance that the official isn't compromised by, or can recover from, a "trusting trust" attack. Although the historical information needed to rebootstrap Rust is mostly complete, it is not trivial to do. Construct a script that can reconstruct the modern Rust compiler from the original OCaml implementation, run it to completion, write a blog post.<br />
! Scripting, build systems<br />
! brson<br />
! acrichto<br />
! <br />
|-<br />
|}<br />
<br />
== Servo ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Off-main thread HTML parsing<br />
| [https://github.com/servo/servo/wiki/Off-main-thread-HTML-parsing-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| Anthony Ramine (nox)<br />
|<br />
|-<br />
| Extend ServiceWorker implementation<br />
| [https://github.com/servo/servo/wiki/More-ServiceWorker-support-project Project page]<br />
| Enthusiasm to learn Rust, comfortable reading/writing JavaScript<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| [https://mozillians.org/en-US/u/jdm/ jdm]<br />
| <br />
|}<br />
<br />
== Security Engineering ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Tor Experiment<br />
| Create a an experimental extension that bundles and launches Tor. Benchmark and report the amount of data transferred, time-to-first-byte, bandwidth differentials.<br />
| Add-On creation experience, Programming in C/Javascript.<br />
| Tom Ritter<br />
| Tom Ritter<br />
|-<br />
| StopTrackware.org<br />
| Create a clearinghouse of trackware lists like StopBadware.org, by creating a Track-the-trackers add-on that uses data [http://www2016.net/proceedings/proceedings/p121.pdf "safeness" approach like Cliqz]<br />
| Add-On creation experience (JavaScript, HTML, CSS, Networking), Cloud Service experience (python or node.js)<br />
| Luke Crouch<br />
| Luke Crouch<br />
|<br />
|}<br />
<br />
== Localization ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title<br />
! Details<br />
! Skills Needed<br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Design screenshot-based localization interface in [https://pontoon.mozilla.org/ Pontoon]<br />
| We'd like to explore the feasibility of screenshot-based localization process in [https://pontoon.mozilla.org/ Pontoon] - Mozilla's translation tool. To provide more translation context, each string will be accompanied by a screenshot showing how the string is used in the application. Additionally, localizers will have the ability to navigate and filter strings by screenshots, as well as preview their translations in the localized screenshots.<br />
| Web standards and Design. Django is a plus.<br />
| [[User:Mathjazz|:mathjazz]]<br />
| [[User:Mathjazz|:mathjazz]]<br />
| <br />
|}<br />
<br />
== Build system ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Security Assurance ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|<br />
TLS 1.3 scanning in Mozilla TLS Observatory<br />
|<br />
[https://github.com/mozilla/tls-observatory Mozilla TLS Observatory] is a hosted service that provides hindsight and compliance checking on the configuration of HTTPS servers. The goal of this project is to improve the service to support scanning TLS 1.3 enabled endpoints, either by improving the [https://github.com/mozilla/cipherscan existing scanner] or writing an entirely new scanner.<br />
|<br />
Programming skills in C, Bash and Go. Strong understanding of TLS and micro-services architectures.<br />
|<br />
Julien Vehent<br />
|<br />
Julien Vehent<br />
|<br />
None<br />
|}<br />
<br />
== WADI ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
|}<br />
<br />
== Mozilla Science Lab ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| science.mozilla.org phase 2<br />
| Start phase 2 of the Mozilla Science Lab website by implementing<br />
* [https://github.com/mozilla/science.mozilla.org/issues/135 github authentication], <br />
* [https://github.com/mozilla/science.mozilla.org/issues/311 members homepage],<br />
* [https://github.com/mozilla/science.mozilla.org/issues/536 improve blog post readability], and add the<br />
* [https://github.com/mozilla/science.mozilla.org/issues/518 ability to join projects]<br />
| js, html, css, react, django, python<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/alanmoo/ alanmoo]<br />
| Detailed list and breakdown of all phase 2 work (only the tasks mentioned in details are expected to be completed by the student):<br />
* Phase 2.1 https://github.com/mozilla/science.mozilla.org/milestone/11<br />
* Phase 2.2 https://github.com/mozilla/science.mozilla.org/milestone/12 <br />
|-<br />
| Paper Badger - badgr migration & publisher integration<br />
| * Issuing badges to credit authors for their work on academic papers.<br />
* In the final steps of our badgekit to Badgr migration, add tests to make sure we have feature parity and make the switch to Badgr in production.<br />
* Implement a [https://github.com/mozillascience/PaperBadger/issues/160 journal submission system integration] with paper badger. This way, journals won't have to manually enter new badges in our system.<br />
| js, Badgr, APIs, authentication, node.js, react<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| [https://mozillians.org/en-US/u/Abby/ abbycabs]<br />
| github: https://github.com/mozillascience/paperbadger<br />
|-<br />
| Study Group participation visualization<br />
| [https://github.com/mozillascience/studyGroup study groups] meet all over the world, running regular events share skills, co-work and create community around open science. Right now there's no easy way to see all the study group events.<br />
* Write a script to collect data for all study group events happening world wide. Each study group is a [https://github.com/mozillascience/studyGroup fork of this template repo]. Each event is an issue in the repo ([https://github.com/UofTCoders/Events/issues example of events as issues]). [https://github.com/mozillascience/studyGroup/blob/gh-pages/scripts/updateCalendar.py here is a script] that collects all events for a specific repo. Please update this to collect data from all forks.<br />
* Visualize this data in a way that shows study group activity all around the world! viz: calendar, map<br />
| js, html, css, python (or another scripting language), data visualization<br />
| [https://mozillians.org/en-US/u/zannah/ Zannah]<br />
| [https://mozillians.org/en-US/u/aurelia/ Aurelia]<br />
| main study group repo: https://github.com/mozillascience/studyGroup<br />
|}<br />
<br />
<br />
== MozVR ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
== Connected Devices ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Reporter<br />
! Mentor(s) <br />
! Comments <br />
|-<br />
|}<br />
<br />
<br />
<br />
== Thimble ==<br />
<br />
{| class="standard-table" border="1" style="border-collapse: collapse"<br />
|-<br />
! Title <br />
! Details <br />
! Skills Needed <br />
! Reporter <br />
! Mentor(s) <br />
! Comments<br />
|-<br />
| Address stability issues within Thimble<br />
| Main task: Get Thimble to write directly to AWS instead of writing to a database first.<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| Full list of issues affecting thimble's stability: https://github.com/mozilla/thimble.mozilla.org/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20label%3A%22good%20first%20bug%22%20-label%3A%22assigned%20to%20contributor%22<br />
|-<br />
| Integrating VR capabilities into Thimble<br />
| Goal: Thimble makes it easy to create multi user VR experiences, allowing the whole classroom to take virtual field trips through student projects.<br />
This is made possible by doing any of these:<br />
* Integration of the A-frame visual inspector, which gives the user lots of control over an object properties in a custom UI<br />
* Custom Quick-Edit UI for different types of objects<br />
* Develop large, visual menu of 3D objects make it easy to build interesting scenes quickly<br />
* A wide range of starter projects<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| This work would be done in parallel with work done by [https://cdot.senecacollege.ca/ CDOT] on making Thimble a VR experience building tool<br />
thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|-<br />
| Thimble and Remote Mentorship<br />
| Goal: Use together.js to enable a multi-user environment on Thimble that is tailored to work as a virtual teaching environment for HTML/CSS/JS<br />
Details:<br />
* Fixing issues in together.js to make it functional<br />
* working on ways to sync state between Thimble's editor and the remote Thimble editor(s<br />
* Syncing filesystems across Thimble editors with WebRTC<br />
| js, html, css<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| [https://mozillians.org/en-US/u/gideon/ gideon]<br />
| thimble repo: https://github.com/mozilla/thimble.mozilla.org<br />
|}</div>Wlachhttps://wiki.mozilla.org/index.php?title=EngineeringProductivity/Projects/Stockwell/Meetings/2017-02-28&diff=1163989EngineeringProductivity/Projects/Stockwell/Meetings/2017-02-282017-02-28T15:51:14Z<p>Wlach: /* Status */ fix</p>
<hr />
<div>= Previous Action Items =<br />
<br />
= Status =<br />
* [wlach] Retriggerable jobs is getting closer ({{Bug|1322433}}) - [https://wiki.mozilla.org/File:Treeherder_retrigger_job_with_options.png screenshot]<br />
** Sort of working, mostly. Some mozharness/taskcluster nits getting in the way of it being ready to ship.<br />
<br />
= Discussion Topics =<br />
* <br />
* <br />
<br />
= New Ideas to investigate =<br />
* <br />
<br />
= Action Items =<br />
* Discussion topics for next time<br />
** <br />
** <br />
*</div>Wlachhttps://wiki.mozilla.org/index.php?title=EngineeringProductivity/Projects/Stockwell/Meetings/2017-02-28&diff=1163988EngineeringProductivity/Projects/Stockwell/Meetings/2017-02-282017-02-28T15:50:16Z<p>Wlach: /* Status */</p>
<hr />
<div>= Previous Action Items =<br />
<br />
= Status =<br />
* [wlach] Retriggerable jobs is getting closer ({{Bug|1322433}})<br />
** Sort of working, mostly. Some mozharness/taskcluster nits getting in the way of it being ready to ship.<br />
[[File:Treeherder_retrigger_job_with_options.png|thumbnail|left]]<br />
<p><br />
<br />
= Discussion Topics =<br />
* <br />
* <br />
<br />
= New Ideas to investigate =<br />
* <br />
<br />
= Action Items =<br />
* Discussion topics for next time<br />
** <br />
** <br />
*</div>Wlach