QA/BrowserID/OPsBuildOut: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
|||
| Line 53: | Line 53: | ||
prod: browserid.org | prod: browserid.org | ||
* I assume then the same clients will now point to the above... | * I assume then the same clients will now point to the above... | ||
== Dev == | == Dev == | ||
| Line 74: | Line 75: | ||
</table> | </table> | ||
== Table for OPs == | == Table for OPs == | ||
| Line 97: | Line 99: | ||
<tr><td valign="top">Other: working with infra-sec on documentation, vuln scans, more reliable CEF logging, and RSBAC rollout</td><td valign="top">OPs</td><td valign="top">in progress</td><td valign="top">none</td></tr> | <tr><td valign="top">Other: working with infra-sec on documentation, vuln scans, more reliable CEF logging, and RSBAC rollout</td><td valign="top">OPs</td><td valign="top">in progress</td><td valign="top">none</td></tr> | ||
</table> | |||
== Quick Summaries: July - Oct == | == Quick Summaries: July - Oct == | ||
Revision as of 17:53, 16 November 2011
Back To ==> Mozilla BT Services
Intro
This is the main QA site for the BrowserID OPs Build-Out. All tasks, projects, status, and other important information related to the Build-Out will be listed here and updated frequently.
- Overall/Master Schedule (details): to be added
- Build-Out Schedule per four environments: November/December
- Deployment Schedule per four environments: November/December
- Load Test Schedule per four environments: November/December
- QA Test Schedule per four environments: November/December
- Missing latest information from #identity: 11/14 and newer...
QA
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Track OPs BrowserID build-out planning and HW orders | jbonacci | in progress | see Bug 697477 |
| BrowserID arch review | jbonacci | completed | https://intranet.mozilla.org/Services/Ops/BrowserID#Architecture |
| Track RPM and deployment work in Dev | jbonacci | in progress | see Bug 695596 |
| Track Dev build-out | jbonacci | in progress | see Bug 695957 |
| Track Stage build-out | jbonacci | in progress | see Bugs 695941, 695944 |
| Track Prod build-out | jbonacci | planned | see Bugs 695941, 695943, 695946, 695955, 695956, 695960, 695972, 698044 |
| Track Test/CI build-out | jbonacci | planned | see Bug 695958 |
| Track DEV load test development | jbonacci | in progress | see Issue 504 |
| Track OPs load test deployment | jbonacci | in progress | see Bug 695952 |
| Track OPs dashboard/monitoring work | jbonacci | in progress | see Bugs 695961, 695963, 695967 |
| Load test tutorial | jbonacci | planned | none |
| Logging for QA work | jbonacci | planned | none |
| Monitoring/dashboards tutorial | jbonacci | planned | none |
| Stability testing Dev env | QA | planned | need help from Dev and OPs |
| Stability testing Stage env | QA | planned | need help from Dev and OPs |
| Load testing Stage env | QA/OPs | planned | none |
| Stability testing Prod env | QA | planned | need help from Dev and OPs |
| Stability testing Test/CI env | QA | planned | need from with Dev and OPs |
| QA node/npm "easy" installs | jbonacci | ongoing | https://wiki.mozilla.org/QA/BrowserID/BrowserID_Basic_Install |
| Complete ID2 configuration | jbonacci/lloyd/petef | in progress | working with Lloyd and Pete on this |
| QA RPM installs to ID2 | jbonacci | ongoing | need RPM info from Dev and OPs |
| QA Test Plan updates | jbonacci | ongoing | https://wiki.mozilla.org/QA/BrowserID/TestPlan |
| QA Test wiki updates | jbonacci | ongoing | https://wiki.mozilla.org/QA/BrowserID/BrowserID_Weekly_Trains_Beta |
| QA internal wiki | jbonacci | planned | TBD |
| Develop Manual Test App | jbonacci/ozten | planned | working with Austin on this |
| Port Unit Tests to Stage | jbonacci/stomlinson | planned | working with Shane on this |
| API automation | QA | planned | TBD |
- How to access the new environments from the "front end"?
- REF: https://intranet.mozilla.org/Services/Ops/BrowserID/ZeusConfig
dev: dev-browserid.services.mozilla.com stage: stage-browserid.services.mozilla.com prod: browserid.org
- I assume then the same clients will now point to the above...
Dev
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Initial plan for Scaling to 1 million users | Dev team | completed | https://github.com/mozilla/browserid/wiki/Scaling-to-1M |
| Metrics dashboard | Dev team | completed | see Bug 690107 |
| Maintain "all-in-one" BrowserID | lloyd | in progress | none |
| GitHub repository reorganization | lloyd | completed | Issue 503: improve repository organization |
| Better logging | lloyd | in progress | GitHub Issues 119, 537, 541, 536, 530, 529, etc. |
| Process splitting: browserid, verifier, keysigner, dbwriter | lloyd | completed | Issue 460: 460 break up browserid into several distinct processes |
| Second client (dev.)(beta.)myfavoritebooze.org | lloyd | completed | support 'keep me signed in' feature |
| Developer Engagement Plan | Dev/PM teams | in progress | https://wiki.mozilla.org/Engagement/Developer_Engagement/BrowserIDPlan |
| Crypto discussions and work | Dev/Security teams | in progress | GitHub issus: various |
| Work with OPs: RPM creation for deployment | lloyd/petef | in progress | Bug 695956 |
| Work with OPs: Dev deployment | lloyd/petef | in progress | Bug 695957 |
| Work with OPs: initial Dev test | lloyd/petef | planned | none |
| Work with OPs: load test code and deploy | lloyd/petef | in progress | Bug 695952, Issue 504: repair load_gen |
| Update GitHub Scaling to 1M | lloyd | in progress | https://github.com/mozilla/browserid/wiki/Scaling-to-1M |
Table for OPs
| Task/Project | Owner | Status | Notes, Links, Tickets, Issues |
| Build-out planning | OPs | in progress | https://intranet.mozilla.org/Services/Ops/BrowserID, 11/12-11/14 posting of all four environments on OPs wiki |
| VM setup/config for Dev and Test/CI | petef | in progress | https://intranet.mozilla.org/Services/Ops/BrowserID |
| HW orders for Prod:SCL2 | OPs | in progress | Bugs 695941, 697477, https://intranet.mozilla.org/Services/Ops/BrowserID |
| HW orders for Prod:PHX1 | OPs | in progress | Bugs 695941, 697477, https://intranet.mozilla.org/Services/Ops/BrowserID, prod scl2 hardware has also landed, still waiting on new load balancers (not a blocker for testing), also waiting for HW for PHX1 |
| HW orders for Stage | OPs | in progress | Bugs 695941, 697477, https://intranet.mozilla.org/Services/Ops/BrowserID, staging/loadtest environment hardware has landed and is mostly built, still waiting on NetOPs |
| Logging | petef | in progress | Bugs 695916, 695963, 695967 |
| Metrics | petef | in progress | Bugs 695916, 695963, 695967 |
| RPM design/build/test | petef/lloyd | in progress | Bug 695956, Latest pull request was 11/14 |
| Set up Load test env | petef | in progress | Bug 695952 |
| Zeus configuration | petef | in progress | Bug 695943 |
| MySQL | petef | in progress | Bug 695972 - browserid: figure out & test mysql master/failover strategy, mysql replication/monitoring strategy fleshed out (doc coming) |
| Dev environment deploy and test | petef/lloyd | in progress | Bug 695957, 11/7-11/14 DEV env build-out and test deployments, on track for week of 11/13 |
| Stage environment deploy and test | petef/lloyd | planned | Bug 695946, on track for week of 11/13 |
| Stage environment load test | petef/QA | planned | none |
| Prod environment deploy and test | petef | planned | Bugs 695946, 695960, 698044 |
| Test/CI environment deploy and test | petef | planned | Bug 695958, Planned for week of 11/20 |
| Other: working with infra-sec on documentation, vuln scans, more reliable CEF logging, and RSBAC rollout | OPs | in progress | none |
Quick Summaries: July - Oct
QA:
- Focus for Summer and Fall
- Weekly deployments to Beta environment
- Verification of deployments to current Production environment
- Test Plan and Weekly Trains wikis
- Tracking Dev activity for scaling to one million users
- Tracking OPs activity for BrowserID environment build-outs
DEV:
- Scalability planning started in the Summer.
- Planning path for security review and production build-out was started in the Summer.
- Metrics dashboard for identity
- Code migration to MySQL
- Command-line load generation and performance analysis
- Debug improvements
- Other areas shared with OPs: monitoring, logging improvements, RPM generation script, schema changes to improve database performance and scalability
OPs:
- BrowserID env designs
- Capacity planning
- Hardware was ordered in October
- Dev environment planning, build-out and configuration was started
- RPM generation work was started
- Other areas of work: node cluster, mysql strategy, monitoring strategy, logging, DB performance, scalability, zeus vips setup, nginx/apache, webhead/secure webhead planning
- Dependent on Dev for keysigner design and split, dbwriter design and split, logging improvements, repository restructuring, process split, code integration for packaging, and the load-gen code
- REF: Bug 695940 - BrowserID production tracking bug
Important Links
- IRC: #identity
- BrowserID QA:
- BrowserID Dev:
- BrowserID OPs:
OPs Tickets
- Bug 695940 - BrowserID production tracking bug
- This is a meta bug for all the other OPs tickets
Other Tickets
- Bug 690107 - Build dashboards for identity
- Bug 644776 - Security Review For BrowserID
- Bug 692247 - Accessibility review for revised design of BrowserID
- Bug 694073 - Need socketlabs SMTP parameters for browserid
- Bug 650863 - SSL cert needed for browserid.org
- Bug 695955 - browserid: write rsbac security policies for all host types
GitHub Issues
- Link for Server issues: https://github.com/mozilla/browserid/issues
- Link for Client issues: https://github.com/lloyd/myfavoritebeer.org/issues
- OPs-related issues in GitHub:
- 560: move bcrypt to webhead
- 561: handle mysql master failover in dbwriter process
- 566: heartbeat should not check backend servers
- 571: update rpm build to match new layout
- 575: [rpmbuild] work around npm bug: re-fetch jwcrypto every time
BrowserID Environments
See the main OPs BrowserID site for full details:
- Dev: https://intranet.mozilla.org/Services/Ops/BrowserID#Development_Environment
- Test/CI: https://intranet.mozilla.org/Services/Ops/BrowserID#Test.2FCI_Environment
- Stage: https://intranet.mozilla.org/Services/Ops/BrowserID#Stage_Environment
- Prod: https://intranet.mozilla.org/Services/Ops/BrowserID#Prod_Environment
Load Testing
- Load_Gen code and test - Dev work is still in progress
- See the following bug and issue:
- Bug 695952 - browserid: loadtest
- Issue 504: repair load_gen
- Load Test hardware (per OPs)
- There are eight dedicated client machines to send load
- Need machine name information
- There is also a dedicated VLAN that does not hit the firewall to push more load, if required.
- There are eight dedicated client machines to send load
- Load Test information per four environments: TBD
Dashboards and Monitoring
- Metrics team links:
- Bug 690107 - Build dashboards for identity
- https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics2&path=identity/&file=identity.wcdf
- Requires LDAP access
- OPs bugs:
- Bug 695961 - browserid: verify statsd collection, build pencil dashboards
- Bug 695963 - browserid: coordinate log collection with metrics
- Bug 695967 - browserid: setup monitoring
- OPs links per four environments: TBD
Bugs vs. GitHub Issues
- TBD
Back To ==> Mozilla BT Services