ReleaseEngineering/Applications/Tooltool: Difference between revisions

no edit summary
m (Point to mozilla-central)
No edit summary
Line 30: Line 30:
   ]
   ]


The simplest usecase for tooltool is to provide a manifest and the url of a tooltool server, and run a "fetch" command to download the files mentioned in the manifest:
The simplest usecase for tooltool is to provide a manifest and the url of a tooltool server (this url is not longer used,the last modified file is from 13-Oct-2015) , and run a "fetch" command to download the files mentioned in the manifest:


   python  tooltool.py fetch --url http://tooltool.pub.build.mozilla.org/temp-sm-stuff -m my-manifest.tt
   python  tooltool.py fetch --url http://tooltool.pub.build.mozilla.org/temp-sm-stuff -m my-manifest.tt
Line 36: Line 36:
If a manifest name is not provided, tooltool will default to manifest.tt.
If a manifest name is not provided, tooltool will default to manifest.tt.


Tooltool will build an url for each of the mentioned files concatenating the server url, the hashing algorithm and the file digest, and try to download.
Tooltool is build on https://api.pub.build.mozilla.org/tooltool/ and to find the uploaded file you can search after : Filename,SHA512 hash,Upload author or Upload message (including bug number).


In this example, the urls will be:
example: search after "mitmproxy"


  http://tooltool.pub.build.mozilla.org/temp-sm-stuff/sha512/58941214a8334331e52114aab851fc3d8d5da5dd14983f933da8735c24b0ddcac134e8f13692553199c4d9a14a4b3188b62878a30b9d696edda1204666b60837
  http://tooltool.pub.build.mozilla.org/temp-sm-stuff/sha512/b2a463249bb3a9e7f2a3604697b000d2393db4f37b623fc099beb8456fbfdb332567013a3131ad138d8633cb19c50a8b77df3990d67500af896cada8b6f698b4
  http://tooltool.pub.build.mozilla.org/temp-sm-stuff/sha512/630d01a329c70aedb66ae7118d12ff7dc6fe06223d1c27b793e1bacc0ca84dd469ec1a6050184f8d9c35a0636546b0e2e5be08d9b51285e53eb1c9f959fef59d
  http://tooltool.pub.build.mozilla.org/temp-sm-stuff/931eb84f798dc9add1a10c7bbd4cc85fe08efda26cac473411638d1f856865524a517209d4c7184d838ee542c8ebc9909dc64ef60f8653a681270ce23524e8e4


After downloading the files, their digest is verified according to the algorithm specified in the manifest, and they are finally renamed according to the filename specified in the manifest.


The tooltool download servers are simply apache server folders with a flat structure (all files are stored at root level with no subfolders).
The tooltool download servers are simply apache server folders with a flat structure (all files are stored at root level with no subfolders).
Line 71: Line 66:


== The tooltool cache ==
== The tooltool cache ==
 
This has been implemented in bug [https://bugzilla.mozilla.org/show_bug.cgi?id=858635 https://bugzilla.mozilla.org/show_bug.cgi?id=858635]
This has been implemented in bug [https://bugzilla.mozilla.org/show_bug.cgi?id=858635 https://bugzilla.mozilla.org/show_bug.cgi?id=858635]
 
All the bellow examples are used for the older packages that are on "http://tooltool.pub.build.mozilla.org/temp-sm-stuff" url.
When connecting to a tooltool server to fetch a file, it is now possible to use a local cache specifying the local cache folder with the -c option, e.g.:
When connecting to a tooltool server to fetch a file, it is now possible to use a local cache specifying the local cache folder with the -c option, e.g.:


Line 118: Line 113:
Next, you will need credentials for the upload.  Tooltool is hosted on RelengAPI, so the credentials take the form of a RelengAPI token.  To get such a token, to to https://api.pub.build.mozilla.org, log in, click on "Tokens", and issue a user token with the permissions ''tooltool.upload.internal'' and/or ''tooltool.upload.public'', depending on the nature of your upload.  If you don't have internal permission, then the option won't be available to you.  When you click "Issue", you'll get a long string which you should copy/paste into a (protected) file on your system, and pass to tooltool.py with ''--authentication-file''.
Next, you will need credentials for the upload.  Tooltool is hosted on RelengAPI, so the credentials take the form of a RelengAPI token.  To get such a token, to to https://api.pub.build.mozilla.org, log in, click on "Tokens", and issue a user token with the permissions ''tooltool.upload.internal'' and/or ''tooltool.upload.public'', depending on the nature of your upload.  If you don't have internal permission, then the option won't be available to you.  When you click "Issue", you'll get a long string which you should copy/paste into a (protected) file on your system, and pass to tooltool.py with ''--authentication-file''.


Once all of that is in place, you're ready to perform the upload.  Just like for a Git or Mercurial commit, you'll need to provide a message to indicate why you're making this upload.  Including a bug reference is always helpful.
Once all of that is in place, you're ready to perform the upload.  Just like for a Git or Mercurial commit, you'll need to provide a message to indicate why you're making this upload.  Including a bug reference is always helpful.Also make sure the "file" is in the same repository from where you run tooltool.
 
    python tooltool.py upload "file" --authentication-file=~/.tooltool-token --message "Bug 1234567: add new frobnicator binaries"


    python tooltool.py upload --authentication-file=~/.tooltool-token --message "Bug 1234567: add new frobnicator binaries"


The tool will begin uploading and, when it's finished, may wait a minute or two longer until the files are verified by RelengAPI.  If your upload fails for some reason, just restart it.  Any files already completely uploaded will not need to be uploaded again.
The tool will begin uploading and, when it's finished, may wait a minute or two longer until the files are verified by RelengAPI.  If your upload fails for some reason, just restart it.  Any files already completely uploaded will not need to be uploaded again.
80

edits