Auto-tools/staging: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Blanked the page)
 
(8 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Tools Staging Environment Booking ==
=== masters ===
{| border="1px" class="sortable" style="border: 1px solid black;"
|-
! Master
! User
! Testing
! Notes
! Dates
! Interruptable
! Waiting
|-
| [http://tools-staging-master.mv.mozilla.com:8010/builders mozilla-tests1] (9010)
| jmaher
| {{bug|619780}}
| talos needs to support environment variables for android
| Dec 16 - Dec 20
| yes
| -
|}


=== slaves ===
001 slaves are attached to mozilla-tests1, 002 slaves are attached to mozilla-tests2.  The slaves themselves can also be removed from the pool and booked individually.
==== currently booked ====
{| border="1px" style="border:1px solid black" class="sortable"
|-
! Slave
! User
! Testing/Notes
! Dates
|-
| -
| -
| -
| -
|}
==== known slaves ====
{| border="1px" style="border:1px solid black" class="sortable"
|-
! Platform
! Slaves
|-
| Linux
| tools-r3-fed-001
|-
| Linux64
| tools-r3-fed64-001
|-
| Mac
| tools-r3-leopard-001
|-
|Mac64
| tools-r3-snow-001
|-
|Win32
| tools-r3-xp-001, tools-r3-w7-001
|-
|Win64
| tools-r3-w764-001
|}
* All 002 machines have been donated to create the addon performance testing pool
== Notes & Best Practices ==
=== restarting buildbot ===
Go to your master configuration path and run
make reconfig
or
make stop; make start
=== restarting buildslaves ===
The waterfall should tell you if builders are offline
* click on the builder on your waterfall to get a list of slaves
* vnc into each slave
On Windows:
* open a terminal
* change to the c:\ directory
* run `buildbot start talos-slave`
=== sending changes ===
run
/builds/buildbot/sendchanges.sh
=== vnc vs. rdc ===
* only log into talos slaves using '''vnc'''
* rdc gives you a remote, virtual console on the box, you end up running performance tests against your home machine's graphics display which is not desired
* to restart the slave, open a terminal and run <tt>buildbot start talos-slave</tt>
* you must use vnc to restart a slave; you can ssh into a slave but the tests require a display.  so use vnc
* don't use the startTalos icon on the desktop for reasons i don't understand
=== test-master1 ===
* uses 001 slaves
* master path : buildbot master configuration
/builds/buildbot/mozilla-tests1
* buildbotcustom path
/tools/buildbotcustom/buildbotcustom
* user configs
/builds/buildbot/user-configs
* waterfall: http://tools-staging-master.mv.mozilla.com:8010/waterfall
* sendchange port: 9010
=== test-master2 ===
* uses 002 slaves
* master path
/builds/buildbot/mozilla-tests2
* buildbotcustom path
/tools/buildbotcustom2/buildbotcustom
* user configs
/builds/buildbot/user-configs2
* waterfall: http://tools-staging-master.mv.mozilla.com:8012/waterfall
* sendchange port: 9012
=== switching users ===
==== create your own user environment ====
* go to the user configs path for your master
* create configs-{your_name}
cp -r configs-clean configs-{your_name}
==== create your own buildbotcustom ====
* go to the buildbotcustom path for your master
* create buildbotcustom-{your_name}
cp -r buildbotcustom-clean buildbotcustom-{your_name}
==== switch master to use your configs/buildbotcustom ====
* book the master you want to use
* go to the directory of your master
* run
./switch-user {your_name}
* A possible doc update on script name and location?:
[root@localhost mozilla-tests2]# pwd
/builds/buildbot/mozilla-tests2
[root@localhost mozilla-tests2]# ./switch_configs.sh jmaher
=== buildbot config files and what they do ===
* you will be working out of
configs-{your_name}/mozilla-tests
* you can switch configs from <tt>/builds/buildbot/mozilla-tests1</tt>
./switch_configs {name}
==== config.py ====
* contains configuration for talos tests
* contains configuration for which talos tests are run on which branches
* shared between production and staging machines
==== staging-config.py ====
* contains graph server configuration (points to graphs-stage)
* contains slave names
** '''NOTE''' have to be updated to point to tools machine names instead of releng stage talos machine names)
==== staging_tests_master_stm{01,02}_localconfig.py ====
* sets slave port number
* sets master waterfall link
** '''NOTE''' has to be updated to point to correct tools set up
* sets active branches + platforms
** can switch on/off branches to narrow down testing to some/all branches
** can switch on/off platforms to narrow testing to some/all OSes
=== how to run custom talos code ===
* talos code is checked out per-run by default
* edit
/tools/buildbotcustom{ ,2}/buildbotcustom-{your-name}/process/factory.py
#self.customTalos = customTalos
self.customTalos = 'talos.zip' #customTalos
* there is already a copy of talos in your user configs that you can edit
/build/buildbot/user-configs{ ,2}/configs-{your_name}/talos
* just package it into and it will be downloaded and unzipped per-run
/build/buildbot/user-configs{ ,2}/configs-{your_name}/talos.zip
=== userland code ===
If you need non shared code, make a directory
/home/cltbld/{your_name}
== Landing Your Changes ==
=== Are you ready to land ===
* Full, green staging run?
* r+ review?
=== Talos Changes ===
* push changes to http://hg.mozilla.org/build/talos/
* pull a fresh copy
* rm -rf .hg
* package into talos.zip
zip -r talos.zip talos/
unzip -l talos.zip
Archive:  talos.zip
  Length    Date  Time    Name
  --------    ----  ----    ----
        0  10-14-10 15:21  talos/
      6618  10-05-10 15:27  talos/remote.config
      7936  10-05-10 15:27  talos/ffprocess_win32.py
    19030  10-14-10 15:20  talos/PerfConfigurator.py
      287  10-05-10 15:27  talos/.hgtags
        0  10-05-10 15:29  talos/page_load_test/
        0  10-05-10 15:27  talos/page_load_test/svg_opacity/
      177  10-05-10 15:27  talos/page_load_test/svg_opacity /svg_opacity.manifest
    191667  10-05-10 15:27  talos/page_load_test/svg_opacity/small-group-opacity-2500.svg
    120159  10-05-10 15:27  talos/page_load_test/svg_opacity/big-optimizable-group-opacity-2500.svg
    33272  10-05-10 15:27  talos/page_load_test/parray.js
        0  10-05-10 15:27  talos/page_load_test/dhtml/
    13272  10-05-10 15:27  talos/page_load_test/dhtml/replaceimages.html
      3409  10-05-10 15:27  talos/page_load_test/dhtml/slidein.html
      4800  10-05-10 15:27  talos/page_load_test/dhtml/movingtext.html
      1425  10-05-10 15:27  talos/page_load_test/dhtml/slidingballs.html
      1529  10-05-10 15:27  talos/page_load_test/dhtml/colorfade.html
      832  10-05-10 15:27  talos/page_load_test/dhtml/textslide.html
    31237  10-05-10 15:27  talos/page_load_test/dhtml/layers1.html
      716  10-05-10 15:27  talos/page_load_test/dhtml/imageslide.html
      830  10-05-10 15:27  talos/page_load_test/dhtml/diagball.html
    31198  10-05-10 15:27  talos/page_load_test/dhtml/layers4.html
      1686  10-05-10 15:27  talos/page_load_test/dhtml/mozilla.html
      2729  10-05-10 15:27  talos/page_load_test/dhtml/fadespacing.html
      995  10-05-10 15:27  talos/page_load_test/dhtml/dhtml.manifest
      1243  10-05-10 15:27  talos/page_load_test/dhtml/meter.html
    31274  10-05-10 15:27  talos/page_load_test/dhtml/layers6.html
    31212  10-05-10 15:27  talos/page_load_test/dhtml/layers2.html
      596  10-05-10 15:27  talos/page_load_test/dhtml/zoom.html
        0  10-05-10 15:27  talos/page_load_test/dhtml/images/
      472  10-05-10 15:27  talos/page_load_test/dhtml/images/element_subnavigatio.gif
      2288  10-05-10 15:27  talos/page_load_test/dhtml/images/img1.jpg
      3329  10-05-10 15:27  talos/page_load_test/dhtml/images/css-icon.jpg
      1144  10-05-10 15:27  talos/page_load_test/dhtml/images/3dball.gif
      3495  10-05-10 15:27  talos/page_load_test/dhtml/images/gecko-icon.jpg
      2401  10-05-10 15:27  talos/page_load_test/dhtml/images/moz.gif
      4016  10-05-10 15:27  talos/page_load_test/dhtml/images/bugs-icon.jpg
      2956  10-05-10 15:27  talos/page_load_test/dhtml/images/dhtml-icon.jpg
      1659  10-05-10 15:27  talos/page_load_test/dhtml/images/welcome-icon.jpg
      2221  10-05-10 15:27  talos/page_load_test/dhtml/images/img2.jpg
      2912  10-05-10 15:27  talos/page_load_test/dhtml/images/credits-icon.jpg
      1593  10-05-10 15:27  talos/page_load_test/dhtml/images/ball.gif
      <snip>
      ...
      </snip>
    19038  10-05-10 15:27  talos/ttest.py
      6615  10-05-10 15:27  talos/cmanager_linux.py
      3516  10-05-10 15:27  talos/post_file.py
  --------                  -------
  29653726                  1284 files
* verification steps
** check file size
** check # of files packaged
* post copy of talos.zip to your people account
** be careful here, don't get it mixed up with an old copy
* file bug for releng to land your new talos.zip into build.mozilla.org (in talos/zips) during their next downtime (downtime might not be required depending on the update)
** provide the md5sum
=== Dirty Profile Changes or other NPOB ===
For "Not Part of Build" changes that can still affect numbers for developers please follow the following steps:
# Write a notification to dev.planning, dev.platform and dev.tree-management to inform them of the change and what to expect
# Arrange for tree-closure with the Tinderbox sheriff of the day
# Make changes
# Monitor the results in case backout is needed, send email to the above-mentioned lists if this needs to happen and repeat steps 1 & 2
=== Pageloder Changes ===
* push changes to http://hg.mozilla.org/build/pageloader/
* pull a fresh copy
* package into pageloader.xpi
cd pageloader
zip -r pageloader.xpi .
$ unzip -l pageloader.xpi
Archive:  pageloader.xpi
  Length    Date  Time    Name
  --------    ----  ----    ----
      3166  09-17-10 08:22  README
        0  09-17-10 08:23  chrome/
    12694  09-17-10 08:23  chrome/pageloader.js
      2525  09-17-10 08:22  chrome/pageloader.xul
      3344  09-17-10 08:22  chrome/quit.js
      7987  09-17-10 08:22  chrome/report.js
      300  09-17-10 08:22  chrome.manifest
        0  09-17-10 08:22  components/
      7416  09-17-10 08:22  components/tp-cmdline.js
  --------                  -------
    37432                  9 files
* post copy of pageloader.xpi your people account
** be careful here, don't get it mixed up with an old copy
* file bug for releng to land your new pageloader.xpi into http://hg.mozilla.org/build/talos/xpis/pageloader.xpi during their next downtime

Latest revision as of 12:46, 5 May 2014

Retrieved from "https://wiki.mozilla.org/index.php?title=Auto-tools/staging&oldid=972981"