NOTE: This is only meant for a quick read and be able to move fast when your previous knowledge to the system is limited.For full and formal documentation please read Mana's Product Signing (credentials required) and Internal Signing page (credentials required).

NOTE: This is exploratory documentation. It could be wrong.

Introduction

Build machines, before they upload their files to FTP or S3, need their build files signed. The per-check-in and nightly builds submit the files that need signing to the signing servers, however, they initially need to request a valid token.

Download token

The builds have a step called "download token". In this step the buildbot master will contact one of the signing servers, receive back a signing token, and download it to the build machine.

Signing

The downloaded token will be used to talk to the signing server to sign the files (the -t flag specifies the location of the token):

python /builds/slave/m-in-l64-000000000000000000000/tools/release/signing/signtool.py --cachedir /builds/slave/m-in-l64-000000000000000000000/signing_cache -t /builds/slave/m-in-l64-000000000000000000000/token -n /builds/slave/m-in-l64-000000000000000000000/nonce -c /builds/slave/m-in-l64-000000000000000000000/tools/release/signing/host.cert -H signing4.srv.releng.scl3.mozilla.com:9110 -H signing5.srv.releng.scl3.mozilla.com:9110 -H signing6.srv.releng.scl3.mozilla.com:9110 -f gpg '../../dist//firefox-29.0a1.en-US.linux-x86_64.checksums'

The signing server will recognize the token as valid since it had issued it earlier. I believe the upload step will rotate through the list of signing servers until it finds the one that will accept the token on-disk.

Known failures

Invalid Token *could* be due to several Redis restart

bug 957627 - Invalid token *could* be due to a redis restart. The issues, in this case, are transient.

2014-01-08 09:25:10,475 - 60573158673390f6896d181698326c76d9a83f22: processing ../../dist//firefox-29.0a1.en-US.linux-x86_64.checksums on https://signing4.srv.releng.scl3.mozilla.com:9110
2014-01-08 09:25:11,115 - 60573158673390f6896d181698326c76d9a83f22: uploading for signing
2014-01-08 09:25:11,849 - 60573158673390f6896d181698326c76d9a83f22: error uploading file for signing: 400 Invalid token

Trouble shooting

Sign into the signing servers and look at the logs.

TODO: Add some docs about this.