Loop/Test/Loadtesting: Difference between revisions

Jump to navigation Jump to search

Loop/Test/Loadtesting (view source)

Revision as of 04:38, 22 May 2015

7,159 bytes removed ,  22 May 2015
move off monitoring
(monitoring section)
(move off monitoring)
Line 203: Line 203:
- register : 4
- register : 4


</pre>
[[#toc]]
= Monitoring Tools (STAGE) =
== Loadtest (loadv1) Test Dashboard ==
* http://loads.services.mozilla.com
[[#toc]]
== Sentry: ==
* https://sentry.shared.us-east-1.stage.mozaws.net
** Getting access to Sentry STAGE - talk to the OPs team
[[#toc]]
== Stackdriver: ==
* https://app.stackdriver.com/
* https://app.stackdriver.com/groups/11042/loop-server/stage-loop
[[#toc]]
== Graphite ==
* https://graphite.shared.us-east-1.stage.mozaws.net
** Graphite > stats > loop-activated-users
** Graphite > stats > loop-call-urls
** User Graphs > not currently defined
[[#toc]]
== Kibana ==
* https://kibana.shared.us-east-1.stage.mozaws.net/#/dashboard/file/landing.json
*  https://kibana.shared.us-east-1.stage.mozaws.net/index.html#/dashboard/file/loop_http_status.json
[[#toc]]
== Heka ==
* https://heka.shared.us-east-1.stage.mozaws.net/
* (or https://heka.shared.us-east-1.stage.mozaws.net/#health)
** PlugIns - Filters
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopBasicMetrics
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopHTTPStatus
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopActiveDailyCallers
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopTotalUsers
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopUrls
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/filters/LoopActiveDailySubscribers
** PlugIns - Outputs
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/outputs/LoopNginxFileOutput
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/outputs/LoopNginxErrorFileOutput
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/outputs/LoopServerFileOutput
** PlugIns - Encoders
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/encoders/LoopNginxFileOutput-ProtobufEncoder
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/encoders/LoopNginxErrorFileOutput-ProtobufEncoder
*** https://heka.shared.us-east-1.stage.mozaws.net/#plugins/encoders/LoopServerFileOutput-ProtobufEncoder
** Sandboxes
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_total_callers.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_cumulative_callers.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_daily_unique_callers.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_daily_unique_subscribers.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_new_callers.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_weekly_recurring.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_fortnightly_recurring.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_monthly_recurring.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_urls_clicked.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopBasicMetrics/outputs/LoopBasicMetrics.loop_urls_generated.json
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopHTTPStatus/outputs/LoopHTTPStatus.HTTPStatus.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopActiveDailyCallers/outputs/LoopActiveDailyCallers.EstimatedActiveDailyCallers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopActiveDailyCallers/outputs/LoopActiveDailyCallers.EstimatedActiveDailyFxOSCallers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopTotalUsers/outputs/LoopTotalUsers.EstimatedTotalUsers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopTotalUsers/outputs/LoopTotalUsers.EstimatedTotalFxOSUsers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopActiveDailySubscribers/outputs/LoopActiveDailySubscribers.EstimatedActiveDailySubscribers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopActiveDailySubscribers/outputs/LoopActiveDailySubscribers.EstimatedActiveDailyFxOSSubscribers.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopUrls/outputs/LoopUrls.All.cbuf
*** https://heka.shared.us-east-1.stage.mozaws.net/#sandboxes/LoopUrls/outputs/LoopUrls.FirefoxOS.cbuf
[[#toc]]
= Monitoring Tools (PRODUCTION) =
== Sentry ==
* http://sentry.shared.us-west-2.prod.mozaws.net
** Getting access to Sentry Prod - talk to the OPs team
** http://sentry.shared.us-west-2.prod.mozaws.net/loop-prod/
** http://sentry.shared.us-west-2.prod.mozaws.net/loop-prod/loop/
[[#top]]
== Stackdriver ==
* https://app.stackdriver.com/groups/11044/loop-server/prod-loop
[[#top]]
== Kibana ==
** Main: https://kibana.shared.us-west-2.prod.mozaws.net
** https://kibana.shared.us-west-2.prod.mozaws.net/index.html#/dashboard/file/loop_http_status.json
** https://kibana.shared.us-west-2.prod.mozaws.net/#/dashboard/temp/QeeHeG7MQoGv4HdKzANCEw
[[#top]]
== Heka ==
* Main: https://heka.shared.us-west-2.prod.mozaws.net/#
* https://heka.shared.us-west-2.prod.mozaws.net/#plugins/filters/LoopActiveDailyUsers
* https://heka.shared.us-west-2.prod.mozaws.net/#plugins/filters/LoopHTTPStatus
* https://heka.shared.us-west-2.prod.mozaws.net/#plugins/filters/MsisdnGatewayHTTPStatus
[[#top]]
== loop-server Dashboard ==
* https://metrics.fxa.us-west-2.prod.mozaws.net/loop-server-dashboard
[[#top]]
== Graphite: ==
* https://graphite.shared.us-west-2.prod.mozaws.net (statsd data)
[[#top]]
== Firefox Input Dashboard: ==
* https://input.mozilla.org/en-US/?product=Loop
[[#top]]
== Pingdom: ==
* loop-client: http://stats.pingdom.com/20dar76w4hmv/1304575
* loop-server: http://stats.pingdom.com/20dar76w4hmv/1304565
[[#top]]
= Log Monitoring =
'''Tail multiple logs'''
<pre>
$ cd /media/ephemeral0/nginx/logs
$ tail -f loop_server.access.log loop_server.error.log  default.access.log default.error.log
</pre>
'''Other logs of interest:'''<br />
For example, for loop-server:<br />
* /var/log/circus.log
* /var/log/loop_err.log
* /var/log/loop_out.log
* /var/log/hekad/loop.stdout.log
* /var/log/hekad/loop.stderr.log
* /var/log/nginx/access.log
* /var/log/nginx/error.log
[[#top]]
= HTTP Access Log - Parsing =
'''unique REST activity with counts'''
<pre>
$ cat /media/ephemeral0/nginx/logs/loop_server.access.log | grep "HTTP/" | awk '{print $6" "$3" "}' | sort | uniq -c
</pre>
'''unique REST activity with counts (from a .gz file)'''
<pre>
$ zgrep -a "HTTP/"  /media/ephemeral0/nginx/logs/loop_server.access.log-20150114.gz |  awk '{print $6" "$3" "}' | sort | uniq -c
</pre>
'''return only the interesting stuff'''
<pre>
$ cat loop_server.access.log | grep -v '200 ' | grep -v '101 ' | grep -v '499 ' | grep -v '201 ' | grep -v '204 '
</pre>
</pre>


Line 448: Line 221:


= Reference =
= Reference =
== Monitoring ==
* [[Loop/Test/Monitoring]]




Rpappalardo
Confirmed users
487

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1075922"

Navigation menu