Auto-tools/Projects/Charts - Revision history

Klahnakoski: simple highlight

2014-12-15T22:35:35Z

simple highlight

← Older revision		Revision as of 22:35, 15 December 2014
Line 6:		Line 6:
	= Hacking/Contribution =		= Hacking/Contribution =

	If you are good at making elegant UI, or are familiar with client side web frameworks, and you like having your work front-and-center (along with all the accolades and "constructive" feedback that comes with it) then this project is for you!		If you are good at making elegant UI, or are familiar with client side web frameworks, '''and''' you like having your work front-and-center (along with all the accolades and "constructive" feedback that comes with it) then this project is for you!

	Contributing to charts.mozilla.org has some distinct benefits over other projects:		Contributing to charts.mozilla.org has some distinct benefits over other projects:

Klahnakoski: /* Web Server */

2014-12-15T22:10:35Z

Web Server

← Older revision		Revision as of 22:10, 15 December 2014
Line 63:		Line 63:
	== Web Server ==		== Web Server ==
	[[image:Auto_tools_Projects_Charts_Architecture.png\|right\|thumb\|200px]]		[[image:Auto_tools_Projects_Charts_Architecture.png\|right\|thumb\|200px]]
	The application itself is served as a set of static html and javascript files from the Mozilla PAAS Stackato servers. There are two versions, Production and Staging; the latter will usually have more features, while being slightly more buggy. Please us the staging server as much as possible: like Firefox Nightly, using staging will help find bugs sooner.		The application itself is served as a set of static html and javascript files from the Mozilla PAAS Stackato servers. There are two versions, Production and Staging; the latter will usually have more features, while being slightly more buggy. Please use the staging server as much as possible: like Firefox Nightly, using staging will help find bugs sooner.

	Code for each version is in a separate branch		Code for each version is in a separate branch

Klahnakoski: more contributor notes

2014-12-15T22:03:59Z

more contributor notes

← Older revision		Revision as of 22:03, 15 December 2014
Line 4:		Line 4:
	This project is highly visible to the FirefoxOS team, and Mozilla in general.		This project is highly visible to the FirefoxOS team, and Mozilla in general.

	= Hacking/~~Contributers~~ =		= Hacking/Contribution =

	~~This~~ project ~~can benefit from your help.~~		If you are good at making elegant UI, or are familiar with client side web frameworks, and you like having your work front-and-center (along with all the accolades and "constructive" feedback that comes with it) then this project is for you!

	Contributing to charts.mozilla.org has some distinct benefits over other projects:		Contributing to charts.mozilla.org has some distinct benefits over other projects:
	* Your work is highly visible to people working at Mozilla		* Your work is highly visible to people working at Mozilla
	* The pages are ~~simple~~ static HTML and Javascript, and ~~have~~ a small learning curve		* The pages are static HTML and Javascript; and can be deployed fast
	* Setting up a development environment is very simple ~~because of this~~ simple static ~~nature~~		* Assuming you already know Javascript; there is a small learning curve.
			* Setting up a development environment is very simple

			== Code ==
			The [https://github.com/mozilla/charts code and setup instructions] can be found on Github. Since these are simple static pages, you will find it quite easy.

			==Contact==
			You can contact me, Kyle Lahnakoski

			:'''IRC:''' ekyle@irc.mozilla.org (in #ateam)
			:'''email:''' klahnakoski@mozilla.com

			I am available to get you started and discuss bus that you may find interesting.

	==Good First Bugs==		==Good First Bugs==
			This is a current list of bugs (from Bugzilla) that have been marked as simple, and exist primarily to ensure you are familiar with Github and the pull request process.

	<div>		<div>
Line 31:		Line 44:

	==All Open Issues==		==All Open Issues==
			This is a list of bugs that are more involved. Please read through them to get an idea of where we would like to go with this project.

	<div>		<div>
	<bugzilla>		<bugzilla>
Line 42:		Line 57:
	</bugzilla>		</bugzilla>
	</div>		</div>


	~~==Contact==~~
	~~You can contact me, Kyle Lahnakoski~~

	~~:'''IRC:''' ekyle@irc.mozilla.org (in #ateam)~~
	~~:'''email:''' klahnakoski@mozilla.com~~

	~~I am available to get you started and discuss bus that you may find interesting.~~

	= Architecture =		= Architecture =

Klahnakoski: formatting

2014-12-15T21:46:31Z

formatting

← Older revision		Revision as of 21:46, 15 December 2014
Line 1:		Line 1:
	= charts.mozilla.org =		= Overview of charts.mozilla.org =

	~~== Overview =~~=
	This project is responsible for presenting the dashboards and details required by management to track progress on the B2G releases and highlight possible bottlenecks in development.		This project is responsible for presenting the dashboards and details required by management to track progress on the B2G releases and highlight possible bottlenecks in development.

	This project is highly visible to the FirefoxOS team, and Mozilla in general.		This project is highly visible to the FirefoxOS team, and Mozilla in general.

	== Hacking/Contributers ==		= Hacking/Contributers =

	This project can benefit from your help.		This project can benefit from your help.
Line 15:		Line 13:
	* Setting up a development environment is very simple because of this simple static nature		* Setting up a development environment is very simple because of this simple static nature

	===Good First Bugs===		==Good First Bugs==

	<div>		<div>
Line 32:		Line 30:
	</div>		</div>

	===All Open Issues===		==All Open Issues==
	<div>		<div>
	<bugzilla>		<bugzilla>
Line 46:		Line 44:


	===Contact===		==Contact==
	You can contact me, Kyle Lahnakoski		You can contact me, Kyle Lahnakoski

Line 54:		Line 52:
	I am available to get you started and discuss bus that you may find interesting.		I am available to get you started and discuss bus that you may find interesting.

	== Architecture ==		= Architecture =
	The '''charts.mozilla.org''' application is a pure Javascript client running on the browser. It accesses the [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] cluster for data.		The '''charts.mozilla.org''' application is a pure Javascript client running on the browser. It accesses the [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] cluster for data.

	=== Web Server ===		== Web Server ==
	[[image:Auto_tools_Projects_Charts_Architecture.png\|right\|thumb\|200px]]		[[image:Auto_tools_Projects_Charts_Architecture.png\|right\|thumb\|200px]]
	The application itself is served as a set of static html and javascript files from the Mozilla PAAS Stackato servers. There are two versions, Production and Staging; the latter will usually have more features, while being slightly more buggy. Please us the staging server as much as possible: like Firefox Nightly, using staging will help find bugs sooner.		The application itself is served as a set of static html and javascript files from the Mozilla PAAS Stackato servers. There are two versions, Production and Staging; the latter will usually have more features, while being slightly more buggy. Please us the staging server as much as possible: like Firefox Nightly, using staging will help find bugs sooner.
Line 73:		Line 71:
	Since the app is entirely client-side, it does not matter where the application is served from (I have various development versions on my people page). If both these servers are down, a simple <code>git clone</code> can allow you to "serve" the app directly from your local filesystem.		Since the app is entirely client-side, it does not matter where the application is served from (I have various development versions on my people page). If both these servers are down, a simple <code>git clone</code> can allow you to "serve" the app directly from your local filesystem.

	=== esFrontline ===		== esFrontline ==
	esFrontline is a simple Python-based proxy to limit ES requests to search requests and limit the indexes exposed. Other than these restrictions, ''this proxy is invisible to client application''. See https://wiki.mozilla.org/BMO/ElasticSearch for more details.		esFrontline is a simple Python-based proxy to limit ES requests to search requests and limit the indexes exposed. Other than these restrictions, ''this proxy is invisible to client application''. See https://wiki.mozilla.org/BMO/ElasticSearch for more details.

	=== ElasticSearch Clusters ===		== ElasticSearch Clusters ==
	Once the application is downloaded it will attempt to contact both the private and public clusters simultaneously; whichever responds will be chosen for all future connections, with preference given to the private cluster. The queries for the dashboard are then sent to the cluster as the dashboard app requires.		Once the application is downloaded it will attempt to contact both the private and public clusters simultaneously; whichever responds will be chosen for all future connections, with preference given to the private cluster. The queries for the dashboard are then sent to the cluster as the dashboard app requires.

	The clusters are configured to accept requests from any client. Hopefully this will promote development of alternative dashboards and charts.		The clusters are configured to accept requests from any client. Hopefully this will promote development of alternative dashboards and charts.

	=== Development ===		== Development ==
	The Development server is responsible for secondary indexes built from the main <code>bug_version</code>. Currently it maintains the hierarchy index for determining recursive dependencies on bugs.		The Development server is responsible for secondary indexes built from the main <code>bug_version</code>. Currently it maintains the hierarchy index for determining recursive dependencies on bugs.

	=== Slowness ===		== Slowness ==
	There are some sources of slowness:		There are some sources of slowness:

Line 108:		Line 106:
	* For problems with the database, e.g. basic framework loads but no data appears, see jakem, cyliang, or fubar in #it on irc.		* For problems with the database, e.g. basic framework loads but no data appears, see jakem, cyliang, or fubar in #it on irc.

	=== Instructions ===		== Instructions ==

	* [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-ObtainingandInstallingtheClient Stackato Instructions on Mana]		* [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-ObtainingandInstallingtheClient Stackato Instructions on Mana]
	* [https://api.paas.allizom.org/console/overview/ Stackato web console ]		* [https://api.paas.allizom.org/console/overview/ Stackato web console ]

	== Past Problems ==		= Past Problems =

	=== CORS ===		== CORS ==
	The charts application makes cross-platform requests: The app is served from charts.mozilla.org and data requested from esfrontline.bugzilla.mozilla.org. This requires the various proxy servers (not shown in architecture) ensure the <code>Access-Control-Allow-Origin</code> HTTP response header be set appropriately. In the past it has been shown this header is stripped from the esFrontline response and set according to Operations' guidelines.		The charts application makes cross-platform requests: The app is served from charts.mozilla.org and data requested from esfrontline.bugzilla.mozilla.org. This requires the various proxy servers (not shown in architecture) ensure the <code>Access-Control-Allow-Origin</code> HTTP response header be set appropriately. In the past it has been shown this header is stripped from the esFrontline response and set according to Operations' guidelines.

	=== Clusters Down or Not Responding ===		== Clusters Down or Not Responding ==
	ElasticSearch is still prone to OutOfMemoryExceptions. Occasionally, this will bring down nodes in the cluster. '''The best solution has always been to reboot the problem nodes''' (or all of them).		ElasticSearch is still prone to OutOfMemoryExceptions. Occasionally, this will bring down nodes in the cluster. '''The best solution has always been to reboot the problem nodes''' (or all of them).

	The chance of data loss is very low: First, the data is replicated 2 times (for a total of three copies). Second, the ETL daemon (responsible for filling the cluster) performs some simple data checks, and writes full history of all changed bugs: Effectively overwriting corruption on any bugs that does occur. Corruption on inactive bugs can linger; assuming corruption exists, and assuming a change happened when the cluster was misbehaving, and assuming the ETL did not detect the misbehavior. Third, there are consistency checks built into MoDevMetrics that monitors or consistency between bug versions, corruption is sometimes detected, and inevitably fixed by the next ETL run.		The chance of data loss is very low: First, the data is replicated 2 times (for a total of three copies). Second, the ETL daemon (responsible for filling the cluster) performs some simple data checks, and writes full history of all changed bugs: Effectively overwriting corruption on any bugs that does occur. Corruption on inactive bugs can linger; assuming corruption exists, and assuming a change happened when the cluster was misbehaving, and assuming the ETL did not detect the misbehavior. Third, there are consistency checks built into MoDevMetrics that monitors or consistency between bug versions, corruption is sometimes detected, and inevitably fixed by the next ETL run.

Klahnakoski: reword

2014-12-15T21:43:40Z

reword

← Older revision		Revision as of 21:43, 15 December 2014
Line 47:		Line 47:

	===Contact===		===Contact===
	~~Feel free to~~ contact me, Kyle Lahnakoski~~, if you have questions.~~		You can contact me, Kyle Lahnakoski
	:'''IRC:''' ekyle@irc.mozilla.org
			:'''IRC:''' ekyle@irc.mozilla.org (in #ateam)
	:'''email:''' klahnakoski@mozilla.com		:'''email:''' klahnakoski@mozilla.com

			I am available to get you started and discuss bus that you may find interesting.

	== Architecture ==		== Architecture ==

Klahnakoski: bug fix

2014-12-15T17:42:20Z

bug fix

← Older revision		Revision as of 17:42, 15 December 2014
Line 17:		Line 17:
	===Good First Bugs===		===Good First Bugs===

			<div>
			<bugzilla>
			{
			"include_fields": "id, summary, status, resolution, assigned_to",
			"resolution":"---",
	"f1":"blocked",		"f1":"blocked",
	"o1":"equals",		"o1":"equals",
Line 23:		Line 28:
	"o2":"anywordssubstr",		"o2":"anywordssubstr",
	"v2":"[good first bug]"		"v2":"[good first bug]"
			}
			</bugzilla>
			</div>

	===All Open Issues===		===All Open Issues===
Line 43:		Line 50:
	:'''IRC:''' ekyle@irc.mozilla.org		:'''IRC:''' ekyle@irc.mozilla.org
	:'''email:''' klahnakoski@mozilla.com		:'''email:''' klahnakoski@mozilla.com


	== Architecture ==		== Architecture ==

Klahnakoski: add good first bugs

2014-12-15T17:41:35Z

add good first bugs

← Older revision		Revision as of 17:41, 15 December 2014
Line 17:		Line 17:
	===Good First Bugs===		===Good First Bugs===

	~~incomplete~~		"f1":"blocked",
			"o1":"equals",
			"v1":1014275,
			"f2":"status_whiteboard",
			"o2":"anywordssubstr",
			"v2":"[good first bug]"


	===~~Current~~ Issues===		===All Open Issues===
	<div>		<div>
	<bugzilla>		<bugzilla>

Klahnakoski: add buglist

2014-12-15T17:39:56Z

add buglist

← Older revision		Revision as of 17:39, 15 December 2014
Line 2:		Line 2:

	== Overview ==		== Overview ==
	~~The '''charts.mozilla.org''' application~~ is ~~a pure Javascript client running~~ on the ~~browser. It accesses the [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] cluster for data~~.		This project is responsible for presenting the dashboards and details required by management to track progress on the B2G releases and highlight possible bottlenecks in development.

	~~==Objective==~~		This project is highly visible to the FirefoxOS team, and Mozilla in general.
	~~The objective of this Charts~~ project is to ~~specifically provide management dashboards~~, ~~but to also exemplify the benefits of our [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] backend~~.

	== Hacking/Contributers ==		== Hacking/Contributers ==

			This project can benefit from your help.

	Contributing to charts.mozilla.org has some distinct benefits over other projects:		Contributing to charts.mozilla.org has some distinct benefits over other projects:
	* Your work is highly visible to people working at Mozilla		* Your work is highly visible to people working at Mozilla
	* The pages are simple static HTML and Javascript, and have a small learning curve		* The pages are simple static HTML and Javascript, and have a small learning curve
	* Setting up a development environment is very simple because of this simple static nature		* Setting up a development environment is very simple because of this simple static nature

			===Good First Bugs===

			incomplete


			===Current Issues===
			<div>
			<bugzilla>
			{
			"include_fields": "id, summary, status, resolution, assigned_to",
			"resolution":"---",
			"f1":"blocked",
			"o1":"equals",
			"v1":1014275
			}
			</bugzilla>
			</div>


	===Contact===		===Contact===
Line 20:		Line 41:

	== Architecture ==		== Architecture ==
			The '''charts.mozilla.org''' application is a pure Javascript client running on the browser. It accesses the [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] cluster for data.

	=== Web Server ===		=== Web Server ===
Line 54:		Line 76:
	* Virtual Machines - The nodes of the ES clusters are hosted on VMs, and may be contributing to some slowness.		* Virtual Machines - The nodes of the ES clusters are hosted on VMs, and may be contributing to some slowness.
	* Code - Nothing is minimized, some pages even pause to load JavaScript dynamically.		* Code - Nothing is minimized, some pages even pause to load JavaScript dynamically.







	= Production Support =		= Production Support =

Klahnakoski: rearrange notes

2014-12-15T17:02:26Z

rearrange notes

@@ Line 2: / Line 2: @@
 == Overview ==
-The '''charts.mozilla.org''' application is pure javascript running on the client side.  It accesses a separate ElasticSearch cluster for data.
+The '''charts.mozilla.org''' application is a pure Javascript client running on the browser.  It accesses the [https://wiki.mozilla.org/BMO/ElasticSearch BMO/ES] cluster for data.
-== Contributers ==
+==Objective==
 Contributing to charts.mozilla.org has some distinct benefits over other projects:
 * Your work is highly visible to people working at Mozilla
@@ Line 10: / Line 13: @@
 * Setting up a development environment is very simple because of this simple static nature
 Feel free to contact me, Kyle Lahnakoski, if you have questions.
-:ekyle@irc.mozilla.org
+:'''IRC:''' ekyle@irc.mozilla.org
-:klahnakoski@mozilla.com
+:'''email:''' klahnakoski@mozilla.com
 == Architecture ==
@@ Line 67: / Line 54: @@
 * Virtual Machines - The nodes of the ES clusters are hosted on VMs, and may be contributing to some slowness.
 * Code - Nothing is minimized, some pages even pause to load JavaScript dynamically.
 == Past Problems ==

Klahnakoski: better docs

2014-10-27T18:31:06Z

better docs

← Older revision		Revision as of 18:31, 27 October 2014
Line 17:		Line 17:

	* For problems with the application itself, e.g. 404 errors, see cturra or others in #paas on irc.		* For problems with the application itself, e.g. 404 errors, see cturra or others in #paas on irc.
	** To do: ~~instructions on restarting~~ the ~~app~~.		** To restart PaaS, use the [https://api.paas.allizom.org/console/overview/ Stackato web console]
			** To upload a new image:
			**# <code>git clone https://github.com/mozilla/charts.git</code>
			**# Use the <code>stackato.yml</code> image to deploy:
			:::: stackato target api.paas.allizom.org
			:::: stackato login
			:::: stackato update

	* For problems with the database, e.g. basic framework loads but no data appears, see jakem, cyliang, or fubar in #it on irc.		* For problems with the database, e.g. basic framework loads but no data appears, see jakem, cyliang, or fubar in #it on irc.

	=== Instructions ===		=== Instructions ===

	* [~~Stackato Instructions on Mana](~~https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-ObtainingandInstallingtheClient)		* [https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=30081453#MozillaPaaS%28Stackato%29-ObtainingandInstallingtheClient Stackato Instructions on Mana]
			* [https://api.paas.allizom.org/console/overview/ Stackato web console ]



	== Architecture ==		== Architecture ==