WebDevITQAMtg09102010: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
* Ways to get Webdev/IT/WebQA on the same page, concerning versions of libraries and binaries (PHP/Python/MySQL, etc.) | *Ways to get Webdev/IT/WebQA on the same page, concerning versions of libraries and binaries (PHP/Python/MySQL, etc.) | ||
** Desired/necessary level of formality: pre-release checklist? What determines "sign off"? | **Desired/necessary level of formality: pre-release checklist? What determines "sign off"? | ||
*** Day before, pre-release meetings | ***Day before, pre-release meetings | ||
*** Webdev lead makes the call and does go/no-go decision | ***Webdev lead makes the call and does go/no-go decision | ||
* Where the responsibility of application/site performance and stress-testing lies, and when it's needed | *Where the responsibility of application/site performance and stress-testing lies, and when it's needed | ||
** Still lies with IT and WebDev | **Still lies with IT and WebDev | ||
* How we can get better at scheduling releases (https://bugzilla.mozilla.org/show_bug.cgi?id=587824 comes to mind)? What's the process? Do we need rules about how many combined large (AMO/SUMO, etc.) and small (Personas, Input) releases we do in a week? | *How we can get better at scheduling releases (https://bugzilla.mozilla.org/show_bug.cgi?id=587824 comes to mind)? What's the process? Do we need rules about how many combined large (AMO/SUMO, etc.) and small (Personas, Input) releases we do in a week? | ||
** Leave it up to WebDev/WebQA to gauge | **Leave it up to WebDev/WebQA to gauge | ||
** OK with 4pm releases, can't do earlier than 4pm | **OK with 4pm releases, can't do earlier than 4pm | ||
* We launched our last AMO release with a brand-new Zeus cluster, but didn't test it prior to launch; maybe Wil knew, but could QA have tested, earlier? (There were no problems, but there could've been.) | *We launched our last AMO release with a brand-new Zeus cluster, but didn't test it prior to launch; maybe Wil knew, but could QA have tested, earlier? (There were no problems, but there could've been.) | ||
* How do we ensure we have user-data backed up, pre-push? | *How do we ensure we have user-data backed up, pre-push? | ||
* If something goes wrong during a push, what are the criteria (I realize this is hard to talk to, in terms of specifics) by which we decide to roll back? What's the allowed user-impacting downtime? Is it the outage window? If we think we can solve a problem, is it fine to extend past the outage window? Do we need to message our users further, with an ETA, etc.? | *If something goes wrong during a push, what are the criteria (I realize this is hard to talk to, in terms of specifics) by which we decide to roll back? What's the allowed user-impacting downtime? Is it the outage window? If we think we can solve a problem, is it fine to extend past the outage window? Do we need to message our users further, with an ETA, etc.? | ||
** we don't truly "panic" until the end of the outage window | **we don't truly "panic" until the end of the outage window | ||
* Outcomes/followups: | *Outcomes/followups: | ||
** | **try to ensure that push bugs have owners earlier ('''mrz''') | ||
** chris is rolling out a security-inventory system (mrz) | **chris is rolling out a security-inventory system ('''mrz''') | ||
*** put the app behind VPN and have WebDev learn how to use it, and do inventories (mrz/morgamic) | ***put the app behind VPN and have WebDev learn how to use it, and do inventories ('''mrz/morgamic''') | ||
** case-by-case basis for sites that need to graduate from VM - | **case-by-case basis for sites that need to graduate from VM -> prod ('''webqa/webdev''') | ||
** set up cron jobs to copy prod DBs - | **set up cron jobs to copy prod DBs -> staging ('''webqa/webdev''' to raise issue to IT when it arises) | ||
** lead developer should know their schemas/run schematic -- to make sure that the schemas are the same between prod/staging (can diff the schemas) (webdev) | **lead developer should know their schemas/run schematic -- to make sure that the schemas are the same between prod/staging (can diff the schemas) ('''webdev''') | ||
** come up with more useful or app-specific outage pages, and a way/person to update that content with a helpful ETA, etc. (owner needed) | **come up with more useful or app-specific outage pages, and a way/person to update that content with a helpful ETA, etc. ('''''owner needed''''') | ||
** try switching the docroot and the DB on a virtual IP for a small site (mrz/webdev) | **try switching the docroot and the DB on a virtual IP for a small site ('''mrz/webdev''') | ||
** can do pushes Tuesday, Wednesday, Thursday (Wed. only if needed) | **can do pushes Tuesday, Wednesday, Thursday (Wed. only if needed) | ||
** small content changes can go whenever | **small content changes can go whenever | ||
** | **enable read-only mode on more sites ('''webdev''') | ||
Latest revision as of 00:18, 11 September 2010
- Ways to get Webdev/IT/WebQA on the same page, concerning versions of libraries and binaries (PHP/Python/MySQL, etc.)
- Desired/necessary level of formality: pre-release checklist? What determines "sign off"?
- Day before, pre-release meetings
- Webdev lead makes the call and does go/no-go decision
- Desired/necessary level of formality: pre-release checklist? What determines "sign off"?
- Where the responsibility of application/site performance and stress-testing lies, and when it's needed
- Still lies with IT and WebDev
- How we can get better at scheduling releases (https://bugzilla.mozilla.org/show_bug.cgi?id=587824 comes to mind)? What's the process? Do we need rules about how many combined large (AMO/SUMO, etc.) and small (Personas, Input) releases we do in a week?
- Leave it up to WebDev/WebQA to gauge
- OK with 4pm releases, can't do earlier than 4pm
- We launched our last AMO release with a brand-new Zeus cluster, but didn't test it prior to launch; maybe Wil knew, but could QA have tested, earlier? (There were no problems, but there could've been.)
- How do we ensure we have user-data backed up, pre-push?
- If something goes wrong during a push, what are the criteria (I realize this is hard to talk to, in terms of specifics) by which we decide to roll back? What's the allowed user-impacting downtime? Is it the outage window? If we think we can solve a problem, is it fine to extend past the outage window? Do we need to message our users further, with an ETA, etc.?
- we don't truly "panic" until the end of the outage window
- Outcomes/followups:
- try to ensure that push bugs have owners earlier (mrz)
- chris is rolling out a security-inventory system (mrz)
- put the app behind VPN and have WebDev learn how to use it, and do inventories (mrz/morgamic)
- case-by-case basis for sites that need to graduate from VM -> prod (webqa/webdev)
- set up cron jobs to copy prod DBs -> staging (webqa/webdev to raise issue to IT when it arises)
- lead developer should know their schemas/run schematic -- to make sure that the schemas are the same between prod/staging (can diff the schemas) (webdev)
- come up with more useful or app-specific outage pages, and a way/person to update that content with a helpful ETA, etc. (owner needed)
- try switching the docroot and the DB on a virtual IP for a small site (mrz/webdev)
- can do pushes Tuesday, Wednesday, Thursday (Wed. only if needed)
- small content changes can go whenever
- enable read-only mode on more sites (webdev)