QA/BrowserID/OPsBuildOut: Difference between revisions
Jump to navigation
Jump to search
(→OPs) |
|||
| (40 intermediate revisions by the same user not shown) | |||
| Line 10: | Line 10: | ||
* QA Test Schedule per four environments: November/December | * QA Test Schedule per four environments: November/December | ||
== QA == | |||
{| width="100%" border="1" style="border-collapse:collapse" | |||
|- | |||
| width="25%" | Task/Project | |||
| width="15%" | Owner | |||
| width="15%" | Status | |||
| width="45%" | Notes, Links, Tickets, Issues | |||
|- | |||
| valign="top" | Track Test/CI build-out | |||
| valign="top" | jbonacci | |||
| valign="top" | Planned | |||
| valign="top" | | |||
*see Bug 695958 | |||
*http://hudson.build.mtv1.svc.mozilla.com/job/browserid-server | |||
* see Bug 695958 | |||
* | |||
|- | |||
| valign="top" | Track new "cross-team env" build-out | |||
| valign="top" | jbonacci | |||
| valign="top" | Planned | |||
| valign="top" | | |||
*This will be a VM environment like old Beta | |||
*This VM env is to be shared by QA teams for BrowserID, AMO, Soup, etc. | |||
|- | |||
| valign="top" | Stability testing Test/CI env | |||
| valign="top" | QA | |||
| valign="top" | Planned | |||
| valign="top" | | |||
* | *need from with Dev and OPs | ||
|- | |||
| valign="top" | Stability testing new "cross-team env" | |||
| valign="top" | QA | |||
| valign="top" | Planned | |||
| valign="top" | | |||
*need from with Dev and OPs | |||
*This will be a VM environment like old Beta | |||
|- | |||
| valign="top" | Track OPs BrowserID build-out planning and HW orders | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bug 697477 - fixed | |||
* Bug | |||
|- | |||
| valign="top" | BrowserID arch review | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID#Architecture | |||
|- | |||
| valign="top" | Track RPM and deployment work in Dev | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bug 695596 | |||
|- | |||
| valign="top" | Track Dev build-out | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bug 695957 - fixed | |||
* Bug | |||
|- | |||
| valign="top" | Track Stage build-out | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bugs 695941 (fixed), 695944 - fixed, 703497 | |||
== Quick Summaries: July - Oct == | |- | ||
| valign="top" | Track Prod build-out | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bugs 695941 (fixed), 695943, 695946, 695955, 695956 (fixed), 695960, 695972, 698044, 703497 | |||
|- | |||
| valign="top" | Track DEV load test development | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Issue 504 | |||
|- | |||
| valign="top" | Track OPs load test deployment | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bug 695952 | |||
|- | |||
| valign="top" | Track OPs dashboard/monitoring work | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bugs 695961, 695963, 695967 | |||
|- | |||
| valign="top" | Unit test tutorial/wiki | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Front-End unit tests, Back-End unit tests, Headless unit tests | |||
*https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install | |||
|- | |||
| valign="top" | Stability testing Dev env | |||
| valign="top" | QA | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Server: https://dev-browserid.services.mozilla.com | |||
*RP: http://beer.mtv1.dev.svc.mozilla.com | |||
|- | |||
| valign="top" | Stability testing Stage env | |||
| valign="top" | QA | |||
| valign="top" | completed | |||
| valign="top" | | |||
*RP1: http://beta.myfavoritebeer.org/ | |||
*RP2: http://beta.myfavoritebooze.org/ | |||
*RP3: http://beta.myfavoriteshow.org/ | |||
*RP4: http://carrera.databits.net:9999/ | |||
*Server: https://diresworb.org/ | |||
|- | |||
| valign="top" | Track Load testing Stage env | |||
| valign="top" | QA/OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|- | |||
| valign="top" | Stability testing Prod env | |||
| valign="top" | QA | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|- | |||
| valign="top" | Load testing Prod env | |||
| valign="top" | QA/Dev/OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|- | |||
| valign="top" | QA node/npm "easy" installs | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Final steps: completed information for installing using unit tests | |||
*https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install | |||
|- | |||
| valign="top" | Discussion of Android compatibility for BrowserID | |||
| valign="top" | QA/Dev | |||
| valign="top" | completed | |||
| valign="top" | | |||
*2.2, 2.3, 3.x and newer vs. supporting older versions | |||
* Percentage-wise (of users) 2.2 and newer seems ok | |||
|- | |||
| valign="top" | Final QA wiki updates | |||
| valign="top" | jbonacci | |||
| valign="top" | completed | |||
| valign="top" | | |||
*https://wiki.mozilla.org/QA/BrowserID/TestPlan | |||
*https://wiki.mozilla.org/QA/BrowserID/BrowserID_Weekly_Trains_Beta | |||
*https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install | |||
*https://wiki.mozilla.org/QA/Browser_Technologies/Services/Releases/BrowserID | |||
*https://intranet.mozilla.org/QA/StageLoadTesting | |||
*https://intranet.mozilla.org/QA/StageAndProdMonitoring | |||
*https://wiki.mozilla.org/QA/BrowserID/OPsBuildOut | |||
*Internal site: TBD | |||
|} | |||
<br> | |||
== Dev == | |||
{| width="100%" border="1" style="border-collapse:collapse" | |||
|- | |||
| width="25%" | Task/Project | |||
| width="15%" | Owner | |||
| width="15%" | Status | |||
| width="45%" | Notes, Links, Tickets, Issues | |||
|- | |||
| valign="top" | Initial plan for Scaling to 1 million users | |||
| valign="top" | Dev team | |||
| valign="top" | completed | |||
| valign="top" | | |||
*https://github.com/mozilla/browserid/wiki/Scaling-to-1M | |||
|- | |||
| valign="top" | Metrics dashboard | |||
| valign="top" | Dev team | |||
| valign="top" | completed | |||
| valign="top" | | |||
*see Bug 690107 | |||
|- | |||
| valign="top" | Maintain "all-in-one" BrowserID | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Git and RPM methods are always available | |||
|- | |||
| valign="top" | GitHub repository re-organization | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Issue 503: improve repository organization | |||
|- | |||
| valign="top" | Better logging | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*GitHub Issues 119, 537, 541, 536, 530, 529, etc. | |||
*Dev needs to work with OPs to get *-error logs, archiving, and log rotation | |||
|- | |||
| valign="top" | Process splitting: browserid, verifier, keysigner, dbwriter | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*GitHub Issue 460 | |||
|- | |||
| valign="top" | Second client (dev.)(beta.)myfavoritebooze.org | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*support 'keep me signed in' feature | |||
|- | |||
| valign="top" | Developer Engagement Plan | |||
| valign="top" | Dev/PM teams | |||
| valign="top" | completed | |||
| valign="top" | | |||
*https://wiki.mozilla.org/Engagement/Developer_Engagement/BrowserIDPlan | |||
|- | |||
| valign="top" | Crypto discussions and work | |||
| valign="top" | Dev/Security teams | |||
| valign="top" | completed | |||
| valign="top" | | |||
*GitHub issues: various | |||
|- | |||
| valign="top" | Work with OPs: RPM creation for deployment | |||
| valign="top" | lloyd/petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695956 - fixed | |||
|- | |||
| valign="top" | Work with OPs: Dev deployment | |||
| valign="top" | lloyd/petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695957 - fixed | |||
|- | |||
| valign="top" | Work with OPs: initial Dev test | |||
| valign="top" | lloyd/petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
* See bullets below | |||
|- | |||
| valign="top" | Work with OPs: code for Stage | |||
| valign="top" | lloyd/petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
* Important open issues for Dev and Stage: | |||
* 504: repair load_gen | |||
* 560: move bcrypt to webhead | |||
* 561: handle mysql master failover in dbwriter process | |||
* 566: heartbeat should not check backend servers | |||
* 582: improved file based configuration | |||
|- | |||
| valign="top" | Work with OPs: load test code and deploy | |||
| valign="top" | lloyd/petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695952 | |||
*Issue 504: repair load_gen | |||
*https://github.com/mozilla/browserid/blob/dev/bin/load_gen | |||
|- | |||
| valign="top" | Update GitHub Scaling to 1M | |||
| valign="top" | lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*https://github.com/mozilla/browserid/wiki/Scaling-to-1M | |||
|- | |||
| valign="top" | Use of node.js v0.6.x | |||
| valign="top" | lloyd, benadida | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|- | |||
| valign="top" | Unit tests, headless front-end tests | |||
| valign="top" | stomlinson, bendadida | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|} | |||
<br> | |||
== OPs == | |||
{| width="100%" border="1" style="border-collapse:collapse" | |||
|- | |||
| width="25%" | Task/Project | |||
| width="15%" | Owner | |||
| width="15%" | Status | |||
| width="45%" | Notes, Links, Tickets, Issues | |||
|- | |||
| valign="top" | Test/CI environment deploy and test | |||
| valign="top" | petef | |||
| valign="top" | Planned | |||
| valign="top" | | |||
*Bug 695958 | |||
*http://hudson.build.mtv1.svc.mozilla.com/job/browserid-server | |||
*Need new DEV public, so just need to get a public VIP/VLAN on our dev/test zeus cluster after that and auto deployment, old dev and move to new dev new dev --> dev.diresworb.org (or equivalent) | |||
*Request sent to NetOPs for external IPs for Dev and Test | |||
*Goal is to have Dev, Test/CI, and Prod up and running and tested by QA by 12/9 | |||
*Stage to be up in same time period, tested, with the addition of load testing | |||
|- | |||
| valign="top" | New "cross-team env" VM deploy and test | |||
| valign="top" | lloyd/petef | |||
| valign="top" | Planned | |||
| valign="top" | | |||
*Need an OPs bug for this | |||
*This Env to be shared by various teams: BrowserID, Soup, AMO, etc. | |||
|- | |||
| valign="top" | Jenkins/Hudson for BrowserID-Server | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*http://hudson.build.mtv1.svc.mozilla.com/job/browserid-server | |||
|- | |||
| valign="top" | Build-out planning | |||
| valign="top" | OPs | |||
| valign="top" | completed<br> | |||
| valign="top" | | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID | |||
*11/12-11/14 posting of all four environments on OPs wiki | |||
|- | |||
| valign="top" | VM setup/config for Dev and Test/CI | |||
| valign="top" | petef | |||
| valign="top" | completed<br> | |||
| valign="top" | | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID | |||
|- | |||
| valign="top" | HW orders for Prod:SCL2 | |||
| valign="top" | OPs | |||
| valign="top" | completed<br> | |||
| valign="top" | | |||
*Bugs 695941 (fixed), 697477 - fixed | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID | |||
|- | |||
| valign="top" | HW orders for Prod:PHX1 | |||
| valign="top" | OPs | |||
| valign="top" | completed<br> | |||
| valign="top" | | |||
*Bugs 695941 (fixed), 697477 - fixed | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID | |||
*prod scl2 hardware has also landed | |||
*still waiting on new load balancers (not a blocker for testing) | |||
*also waiting for HW for PHX1 | |||
|- | |||
| valign="top" | HW orders for Stage | |||
| valign="top" | OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bugs 695941 (fixed), 697477 - fixed | |||
*https://intranet.mozilla.org/Services/Ops/BrowserID | |||
*staging/loadtest environment hardware has landed and is mostly built | |||
*still waiting on NetOPs | |||
|- | |||
| valign="top" | Logging | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bugs 695916, 695963, 695967 | |||
*/var/log/browserid on webheads, secure webheads, keysigners | |||
*/var/log/zeus on Zeus servers | |||
* Still need log archiving and rotation | |||
* Still need *-error logs | |||
* TBD for database | |||
|- | |||
| valign="top" | Metrics | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bugs 695916, 695963, 695967 | |||
|- | |||
| valign="top" | RPM design/build/test | |||
| valign="top" | petef/lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695956 - fixed | |||
*Latest pull request was 11/14 | |||
|- | |||
| valign="top" | Set up Load test env | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695952 | |||
|- | |||
| valign="top" | Monitoring/Dashboards | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*All in Graphite/Pencil | |||
|- | |||
| valign="top" | Zeus configuration | |||
| valign="top" | petef/atoll | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bugs 695943, 705922, 705922, 706408, 706410, 706411, 704638, 708321 | |||
|- | |||
| valign="top" | MySQL | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695972 | |||
*mysql replication/monitoring strategy fleshed out (doc coming) | |||
*DB/MySQL failover work, Bug 695972 | |||
|- | |||
| valign="top" | Initial Dev environment deploy and test | |||
| valign="top" | petef/lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695957 - fixed | |||
*11/7-11/14 DEV env build-out and test deployments | |||
*First official deploymant: train-2011.11.17 | |||
*Dev Server: https://dev-browserid.services.mozilla.com | |||
*Dev RP (temp): http://beer.mtv1.dev.svc.mozilla.com | |||
*OPs needs to configure persistence on the test beer site otherwise beer names are not sticking across sessions | |||
*Need to see what happens on Stage (once it is up and running again) | |||
*OPs needs to puppetize and config mongodb on Dev, so my guess is Stage will need this also | |||
*Request sent to NetOPs for external IPs for Dev and Test | |||
*Goal is to have Dev, Test/CI, and Prod up and running and tested by QA by 12/9 | |||
*Stage to be up in same time period, tested, with the addition of load testing | |||
|- | |||
| valign="top" | Initial Stage environment deploy and test | |||
| valign="top" | petef/lloyd | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695946, 703497, 695944 - fixed | |||
*First official deploymant: train-2011.11.17 | |||
*Stage Server: https://stage-browserid.services.mozilla.com | |||
*Stage RP (temp): http://carrera.databits.net:9999 | |||
*Delayed from last week: dbwriter, networking and our new load balancers | |||
*Stage still needs log rotation and archiving | |||
*Goal is to have Dev, Test/CI, and Prod up and running and tested by QA by 12/9 | |||
*Stage to be up in same time period, tested, with the addition of load testing | |||
|- | |||
| valign="top" | Stage environment load test | |||
| valign="top" | petef/QA | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Planned: benchmarks and performance in Stage. | |||
*QA needs more detailed instructions for running load_gen off of the "client" load generators. | |||
|- | |||
| valign="top" | Prod environment deploy, test, and load test | |||
| valign="top" | petef | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bugs 695946, 695960, 698044, 705922, 703497 | |||
*Goal is to have Dev, Test/CI, and Prod up and running and tested by QA by 12/9 | |||
*Stage to be up in same time period, tested, with the addition of load testing | |||
|- | |||
| valign="top" | Other: working with infra-sec on documentation, vuln scans, more reliable CEF logging, heartbeats | |||
| valign="top" | OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*none | |||
|- | |||
| valign="top" | Puppet configuration (transferable to Stage) | |||
| valign="top" | OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695956 - fixed | |||
|- | |||
| valign="top" | RSBAC, crypto, ACLs work | |||
| valign="top" | OPs | |||
| valign="top" | completed | |||
| valign="top" | | |||
*Bug 695955 | |||
*Completed on Dev | |||
*Planned for Stage and then Prod | |||
|} | |||
<br> | |||
== Quick Summaries/History: July - Oct == | |||
QA: | QA: | ||
* Focus for Summer and Fall | * Focus for Summer and Fall | ||
| Line 229: | Line 604: | ||
* Bug 650863 - SSL cert needed for browserid.org | * Bug 650863 - SSL cert needed for browserid.org | ||
* Bug 695955 - browserid: write rsbac security policies for all host types | * Bug 695955 - browserid: write rsbac security policies for all host types | ||
* Bug 705023 - Set browserid's user UID to something lower than 500 | |||
* Bug 705033 - Change the way nodejs starts browserid (no functionality change) | |||
* Bug 705922 - zlb1/2/3.pub.scl2.svc.m.c: please kickstart, yum update, reboot, puppetize | |||
* Bug 706408 - monitor zeus log sync queue size | |||
* Bug 706410 - deploy zeus log sync ssh key to zlb*.pub.scl2.svc | |||
* Bug 706411 - fix zeus log sync script to report rsync errors via cronmail | |||
* Bug 704638 - deploy zlb*.pub.scl2 | |||
== GitHub Issues == | == GitHub Issues == | ||
Latest revision as of 19:51, 19 December 2011
Back To ==> Mozilla BT Services
Intro
This is the main QA site for the BrowserID OPs Build-Out. All tasks, projects, status, and other important information related to the Build-Out will be listed here and updated frequently.
- Overall/Master Schedule (details): to be added
- Build-Out Schedule per four environments: November/December
- Deployment Schedule per four environments: November/December
- Load Test Schedule per four environments: November/December
- QA Test Schedule per four environments: November/December
QA
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Track Test/CI build-out | jbonacci | Planned | |
| Track new "cross-team env" build-out | jbonacci | Planned |
|
| Stability testing Test/CI env | QA | Planned |
|
| Stability testing new "cross-team env" | QA | Planned |
|
| Track OPs BrowserID build-out planning and HW orders | jbonacci | completed |
|
| BrowserID arch review | jbonacci | completed | |
| Track RPM and deployment work in Dev | jbonacci | completed |
|
| Track Dev build-out | jbonacci | completed |
|
| Track Stage build-out | jbonacci | completed |
|
| Track Prod build-out | jbonacci | completed |
|
| Track DEV load test development | jbonacci | completed |
|
| Track OPs load test deployment | jbonacci | completed |
|
| Track OPs dashboard/monitoring work | jbonacci | completed |
|
| Unit test tutorial/wiki | jbonacci | completed |
|
| Stability testing Dev env | QA | completed | |
| Stability testing Stage env | QA | completed | |
| Track Load testing Stage env | QA/OPs | completed |
|
| Stability testing Prod env | QA | completed |
|
| Load testing Prod env | QA/Dev/OPs | completed |
|
| QA node/npm "easy" installs | jbonacci | completed |
|
| Discussion of Android compatibility for BrowserID | QA/Dev | completed |
|
| Final QA wiki updates | jbonacci | completed |
|
Dev
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Initial plan for Scaling to 1 million users | Dev team | completed | |
| Metrics dashboard | Dev team | completed |
|
| Maintain "all-in-one" BrowserID | lloyd | completed |
|
| GitHub repository re-organization | lloyd | completed |
|
| Better logging | lloyd | completed |
|
| Process splitting: browserid, verifier, keysigner, dbwriter | lloyd | completed |
|
| Second client (dev.)(beta.)myfavoritebooze.org | lloyd | completed |
|
| Developer Engagement Plan | Dev/PM teams | completed | |
| Crypto discussions and work | Dev/Security teams | completed |
|
| Work with OPs: RPM creation for deployment | lloyd/petef | completed |
|
| Work with OPs: Dev deployment | lloyd/petef | completed |
|
| Work with OPs: initial Dev test | lloyd/petef | completed |
|
| Work with OPs: code for Stage | lloyd/petef | completed |
|
| Work with OPs: load test code and deploy | lloyd/petef | completed |
|
| Update GitHub Scaling to 1M | lloyd | completed | |
| Use of node.js v0.6.x | lloyd, benadida | completed |
|
| Unit tests, headless front-end tests | stomlinson, bendadida | completed |
|
OPs
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Test/CI environment deploy and test | petef | Planned |
|
| New "cross-team env" VM deploy and test | lloyd/petef | Planned |
|
| Jenkins/Hudson for BrowserID-Server | petef | completed | |
| Build-out planning | OPs | completed |
|
| VM setup/config for Dev and Test/CI | petef | completed |
|
| HW orders for Prod:SCL2 | OPs | completed |
|
| HW orders for Prod:PHX1 | OPs | completed |
|
| HW orders for Stage | OPs | completed |
|
| Logging | petef | completed |
|
| Metrics | petef | completed |
|
| RPM design/build/test | petef/lloyd | completed |
|
| Set up Load test env | petef | completed |
|
| Monitoring/Dashboards | petef | completed |
|
| Zeus configuration | petef/atoll | completed |
|
| MySQL | petef | completed |
|
| Initial Dev environment deploy and test | petef/lloyd | completed |
|
| Initial Stage environment deploy and test | petef/lloyd | completed |
|
| Stage environment load test | petef/QA | completed |
|
| Prod environment deploy, test, and load test | petef | completed |
|
| Other: working with infra-sec on documentation, vuln scans, more reliable CEF logging, heartbeats | OPs | completed |
|
| Puppet configuration (transferable to Stage) | OPs | completed |
|
| RSBAC, crypto, ACLs work | OPs | completed |
|
Quick Summaries/History: July - Oct
QA:
- Focus for Summer and Fall
- Weekly deployments to Beta environment
- Verification of deployments to current Production environment
- Test Plan and Weekly Trains wikis
- Tracking Dev activity for scaling to one million users
- Tracking OPs activity for BrowserID environment build-outs
DEV:
- Scalability planning started in the Summer.
- Planning path for security review and production build-out was started in the Summer.
- Metrics dashboard for identity
- Code migration to MySQL
- Command-line load generation and performance analysis
- Debug improvements
- Other areas shared with OPs: monitoring, logging improvements, RPM generation script, schema changes to improve database performance and scalability
OPs:
- BrowserID env designs
- Capacity planning
- Hardware was ordered in October
- Dev environment planning, build-out and configuration was started
- RPM generation work was started
- Other areas of work: node cluster, mysql strategy, monitoring strategy, logging, DB performance, scalability, zeus vips setup, nginx/apache, webhead/secure webhead planning
- Dependent on Dev for keysigner design and split, dbwriter design and split, logging improvements, repository restructuring, process split, code integration for packaging, and the load-gen code
- REF: Bug 695940 - BrowserID production tracking bug
Important Links
- IRC: #identity
- BrowserID QA:
- BrowserID Dev:
- BrowserID OPs:
OPs Tickets
- Bug 695940 - BrowserID production tracking bug
- This is a meta bug for all the other OPs tickets
Other Tickets
- Bug 690107 - Build dashboards for identity
- Bug 644776 - Security Review For BrowserID
- Bug 692247 - Accessibility review for revised design of BrowserID
- Bug 694073 - Need socketlabs SMTP parameters for browserid
- Bug 650863 - SSL cert needed for browserid.org
- Bug 695955 - browserid: write rsbac security policies for all host types
- Bug 705023 - Set browserid's user UID to something lower than 500
- Bug 705033 - Change the way nodejs starts browserid (no functionality change)
- Bug 705922 - zlb1/2/3.pub.scl2.svc.m.c: please kickstart, yum update, reboot, puppetize
- Bug 706408 - monitor zeus log sync queue size
- Bug 706410 - deploy zeus log sync ssh key to zlb*.pub.scl2.svc
- Bug 706411 - fix zeus log sync script to report rsync errors via cronmail
- Bug 704638 - deploy zlb*.pub.scl2
GitHub Issues
- Link for Server issues: https://github.com/mozilla/browserid/issues
- Link for Client issues: https://github.com/lloyd/myfavoritebeer.org/issues
- OPs-related issues in GitHub:
- 560: move bcrypt to webhead
- 561: handle mysql master failover in dbwriter process
- 566: heartbeat should not check backend servers
- 571: update rpm build to match new layout
- 575: [rpmbuild] work around npm bug: re-fetch jwcrypto every time
BrowserID Environments
See the main OPs BrowserID site for full details:
- Dev: https://intranet.mozilla.org/Services/Ops/BrowserID#Development_Environment
- Test/CI: https://intranet.mozilla.org/Services/Ops/BrowserID#Test.2FCI_Environment
- Stage: https://intranet.mozilla.org/Services/Ops/BrowserID#Stage_Environment
- Prod: https://intranet.mozilla.org/Services/Ops/BrowserID#Prod_Environment
Load Testing
- Load_Gen code and test - Dev work is still in progress
- See the following bug and issue:
- Bug 695952 - browserid: loadtest
- Issue 504: repair load_gen
- Load Test hardware (per OPs)
- There are eight dedicated client machines to send load
- Need machine name information
- There is also a dedicated VLAN that does not hit the firewall to push more load, if required.
- There are eight dedicated client machines to send load
- Load Test information per four environments: TBD
Dashboards and Monitoring
- Metrics team links:
- Bug 690107 - Build dashboards for identity
- https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics2&path=identity/&file=identity.wcdf
- Requires LDAP access
- OPs bugs:
- Bug 695961 - browserid: verify statsd collection, build pencil dashboards
- Bug 695963 - browserid: coordinate log collection with metrics
- Bug 695967 - browserid: setup monitoring
- OPs links per four environments: TBD
Bugs vs. GitHub Issues
- TBD
Back To ==> Mozilla BT Services