Confirmed users
513
edits
Auto-tools/Projects/Charts: Difference between revisions
Jump to navigation
Jump to search
→Past Problems
Klahnakoski (talk | contribs) |
Klahnakoski (talk | contribs) |
||
| Line 43: | Line 43: | ||
=== CORS === | === CORS === | ||
The charts application makes cross-platform requests: The app is served from charts.mozilla.org and data requested from esfrontline.bugzilla.mozilla.org. This requires the various proxy servers (not shown in architecture) ensure the <code>Access-Control-Allow-Origin</code> HTTP response header be set appropriately. In the past it has been shown this header is stripped from the esFrontline response and set according to Operations' guidelines. | |||
The | === Clusters Down or Not Responding === | ||
ElasticSearch is still prone to OutOfMemoryExceptions. Occasionally, this will bring down nodes in the cluster. '''The best solution has always been to reboot the problem nodes''' (or all of them). | |||
The chance of data loss is very low: First, the data is replicated 2 times (for a total of three copies). Second, the ETL daemon (responsible for filling the cluster) performs some simple data checks, and writes full history of all changed bugs: Effectively overwriting corruption on any bugs that does occur. Corruption on inactive bugs can linger; assuming corruption exists, and assuming a change happened when the cluster was misbehaving, and assuming the ETL did not detect the misbehavior. Third, there are consistency checks built into MoDevMetrics that monitors or consistency between bug versions, corruption is sometimes detected, and inevitably fixed by the next ETL run. | |||