Stage Migration Notes
Currently, contributors and Tinderbox machines upload directly to surf (CNAMEed to stage.m.o). manna (CNAMEd to ftp.m.o) pulls these builds from surf via rsync at set intervals.
surf is responsible for virus scanning and verifying other build-related issues.
Both surf and manna are constantly dangerously low on diskspace; the goals of this migration are to:
- Solve this continuing disk space problem
- Ensure that bits are virus scanned and otherwise verified before reaching the mirrors, which is currently not the case. Infected bits will eventually get removed from ftp.m.o/mirrors, but it's not instantaneous.
On the evening of Thursday, 14 September, from 6 pm to midnight PDT:
- surf/stage will be moved inside the firewall and become responsible for virus scanning and verifying all bits that go to the mirror farm/ftp.m.o. Because of this, access will be restricted to this host.
- surf/stage will have a read/write mount to shared, central storage.
- manna will be renamed stage (as a CNAME), and will mount this shared, central storage read-only. This shared, central storage will become the new definitive copy of ftp.m.o.
- Some tinderboxen will change to publish their builds to surf
- Bits in the archive.m.o rsync module will move entirely to the shared, central storage.
- Bits in the release.m.o rsync module will continue to exist on surf and the shared central storage
- surf will pull bits onto its local disk via a scheduled process and perform various verification steps on them (virus scanning, permissions checks, etc.); when these checks pass, they will be uploaded to the shared, central storage, and immediately appear on the ftp mirrors/ftp.m.o.
How These Changes Affect External Contributors
- External contributors may have to accept a new SSH host key; this key will be published after the switchover is complete, so contributors can verify it before accepting it
- Access to surf will be limited
- As part of this plan, we will be auditing access to manna; if access is removed, but it shouldn't have been, please email email@example.com.
- Builds may be spotty on Friday; we'll do our best to ensure the builds get uploaded as expected.
This is a rough, ordered timeline of the steps necessary to fulfill the requirements.
To be integrated into this process
- Transfer home directories for contributors from surf to manna