ReleaseEngineering/Applications/Tooltool: Difference between revisions

Moved "How to upload" higher to make it easier to find.
(The "file" argument in tooltool upload command is not needed as files are being uploaded from manifest.tt ( https://github.com/mozilla/build-tooltool/blob/master/tooltool.py#L933-L942 ))
(Moved "How to upload" higher to make it easier to find.)
Line 62: Line 62:


The two most basic commands list a manifest and validate the local files against the manifest.  The list command lists out all of the files in the manifest as well as whether they are present and/or valid.  The return code from listing is zero unless there was an error in listing the files.  Absent or invalid files will still result in an exit code of zero if there was no error in the listing process.  The validate command is used to check if all the files in the manifest are present and valid.  The exit code for validating is zero if all files in the manifest are present and their hash matches the manifest.  It is non-zero if any file is missing locally or the file does not have the same hash as the manifest.
The two most basic commands list a manifest and validate the local files against the manifest.  The list command lists out all of the files in the manifest as well as whether they are present and/or valid.  The return code from listing is zero unless there was an error in listing the files.  Absent or invalid files will still result in an exit code of zero if there was no error in the listing process.  The validate command is used to check if all the files in the manifest are present and valid.  The exit code for validating is zero if all files in the manifest are present and their hash matches the manifest.  It is non-zero if any file is missing locally or the file does not have the same hash as the manifest.
= How To Upload To Tooltool  =
{{note|Make sure you're using the latest tooltool.py from https://github.com/mozilla/build-tooltool}}
First, plan out your upload.  Are all of the files you need to upload public?  If not, tooltool also offers storage of "internal" files, which are not made publicly available but can be downloaded by those with proper permissions.  Internal files might be under a non-redistribution license (e.g., Google SDKs, Microsoft DLLs), but '''must not include secrets''' such as passwords or private keys.
Use the `tooltool.py` client to build a manifest containing the files you would like to upload, annotating each file with its visibility level:
    python tooltool.py add --visibility public gcc.tar.gz
Next, you will need credentials for the upload.  Tooltool is hosted on RelengAPI, so the credentials take the form of a RelengAPI token.  To get such a token, to to https://api.pub.build.mozilla.org, log in, click on "Tokens", and issue a user token with the permissions ''tooltool.upload.internal'' and/or ''tooltool.upload.public'', depending on the nature of your upload.  If you don't have internal permission, then the option won't be available to you.  When you click "Issue", you'll get a long string which you should copy/paste into a (protected) file on your system, and pass to tooltool.py with ''--authentication-file''.
Once all of that is in place, you're ready to perform the upload.  Just like for a Git or Mercurial commit, you'll need to provide a message to indicate why you're making this upload.  Including a bug reference is always helpful.Also make sure the "file" is in the same repository from where you run tooltool.
    python tooltool.py upload --authentication-file=~/.tooltool-token --message "Bug 1234567: add new frobnicator binaries"
The tool will begin uploading and, when it's finished, may wait a minute or two longer until the files are verified by RelengAPI.  If your upload fails for some reason, just restart it.  Any files already completely uploaded will not need to be uploaded again.
You can view the available files and uploads at https://api.pub.build.mozilla.org/tooltool.


= Other tooltool features =
= Other tooltool features =
Line 100: Line 121:
* tooltool_wrapper.sh: http://hg.mozilla.org/build/tools/file/327616201e98/scripts/tooltool/tooltool_wrapper.sh
* tooltool_wrapper.sh: http://hg.mozilla.org/build/tools/file/327616201e98/scripts/tooltool/tooltool_wrapper.sh
* TooltoolMixin class for mozharness calls: https://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/mozharness/mozilla/tooltool.py (despite being an Homonym of tooltool itself, this is a python module defining TooltoolMixin class to be used in mozharness calls).
* TooltoolMixin class for mozharness calls: https://hg.mozilla.org/mozilla-central/file/default/testing/mozharness/mozharness/mozilla/tooltool.py (despite being an Homonym of tooltool itself, this is a python module defining TooltoolMixin class to be used in mozharness calls).
= How To Upload To Tooltool  =
{{note|Make sure you're using the latest tooltool.py from https://github.com/mozilla/build-tooltool}}
First, plan out your upload.  Are all of the files you need to upload public?  If not, tooltool also offers storage of "internal" files, which are not made publicly available but can be downloaded by those with proper permissions.  Internal files might be under a non-redistribution license (e.g., Google SDKs, Microsoft DLLs), but '''must not include secrets''' such as passwords or private keys.
Use the `tooltool.py` client to build a manifest containing the files you would like to upload, annotating each file with its visibility level:
    python tooltool.py add --visibility public gcc.tar.gz
Next, you will need credentials for the upload.  Tooltool is hosted on RelengAPI, so the credentials take the form of a RelengAPI token.  To get such a token, to to https://api.pub.build.mozilla.org, log in, click on "Tokens", and issue a user token with the permissions ''tooltool.upload.internal'' and/or ''tooltool.upload.public'', depending on the nature of your upload.  If you don't have internal permission, then the option won't be available to you.  When you click "Issue", you'll get a long string which you should copy/paste into a (protected) file on your system, and pass to tooltool.py with ''--authentication-file''.
Once all of that is in place, you're ready to perform the upload.  Just like for a Git or Mercurial commit, you'll need to provide a message to indicate why you're making this upload.  Including a bug reference is always helpful.Also make sure the "file" is in the same repository from where you run tooltool.
    python tooltool.py upload --authentication-file=~/.tooltool-token --message "Bug 1234567: add new frobnicator binaries"
The tool will begin uploading and, when it's finished, may wait a minute or two longer until the files are verified by RelengAPI.  If your upload fails for some reason, just restart it.  Any files already completely uploaded will not need to be uploaded again.
You can view the available files and uploads at https://api.pub.build.mozilla.org/tooltool.


= Tooltool uploads: the old school =
= Tooltool uploads: the old school =
Confirmed users
67

edits