NSS NISCC test setup on Linux

This page describes how to setup a Linux computer to run the NISCC test suite against NSS. It involves building the NSS software in two variations, one standard build, and another "hacked" build that will deliberately allow to work with corrupt data. The intention of the test is to ensure that no bad data is able to trigger a crash in NSS.

The test involves processing a large set of data, which currently isn't available publicly. Please contact the NSS team if you wish to obtain the data.

It is recommended to run this test suite at least once per week against the latest stable release of NSS. These instructions work on a Linux system.

If you're looking for a system to run these tests, a good candidate is any of the other Linux systems used to run continuous NSS testing. These instructions will use a system that has been prepared according to the instruction in page NSS:LinuxTinderbox.

Let's ensure that a cron daemon is available

 yum install cronie
 service crond start

Obtain the NISCC test data. This is a huge set of data. It consists of approximately 2 million files, using nearly 20 GB on a standard filesystem. However, a SquashFS image is available that is just 530 MB in size, and it can be mounted read-only on a Linux system. That's the recommended setup.

Obtain file niscc.sqsh and move it to /home/niscc.sqsh

Edit /etc/fstab and add:

 /home/niscc.sqsh   /niscc   squashfs   ro,loop   0 2

Prepare a mountpoint and mount the filesystem

 mkdir /niscc
 mount /niscc

The NISCC test data uses certificates that have already expired. In order to avoid early exiting of the tests because of the expiration, we'll use the libfaketime library (version 0.9.1 was used, other versions are untested).

 useradd -m niscc
 su - niscc
 mkdir ~/install
 cd ~/install
 wget http://www.code-wizards.com/projects/libfaketime/libfaketime-0.9.1.tar.gz
 tar xzf libfaketime-0.9.1.tar.gz
 cd libfaketime-0.9.1

We'll install libfaketime for the user account (not globally).

• Edit the Makefile and disable (or remove) the final three targets, the ones that install to share/doc.
• Edit man/Makefile and src/Makefile and in both files change PREFIX to /home/niscc

Now build and install

 make
 make install

and prepare the directory where our test scripts will live

 cd
 mkdir run
 cd run

Obtain script run_niscc.sh from the NSS source control system, find it in directory mozilla/security/nss/tests and copy it to /home/niscc/run

Edit the script, find the line containing MAIL_COMMAND and adjust it accordingly to enable the script to send result email. For example, you could use

 export MAIL_COMMAND=${MAIL_COMMAND:-"mailx -S smtp=smtp://my.smtp.server:25 -r senderaddress+niscc@mydomain.com"}

The script supports a variety of options. We'll create a wrapper script with all preferences. Create file /home/niscc/run/run with the following contents:

 #!/bin/sh
 cd /home/niscc/run
 ./run_niscc.sh --niscc-home /niscc --nss ~/run/cvs/nss --nss-hack ~/run/cvs/nss_hack --out ~/run/out --log-store --libfaketime ~/lib/faketime/libfaketimeMT.so.1 --mail recipient-email-address-for-reports@domain.com

You might want to create another variation of the wrapper run script, to make sure your environment is correct, without having to wait hours for all of the test to complete.

Copy file run to file run-only-smallset, edit file run-only-smallset, append another option: --smallset

Ensure that files are executable

 chmod +x /home/niscc/run/run*

You're ready to execute it for the first time. You might want to open a screen session, then use

 ./run-only-smallset

(In order to test that detection of crashes works correctly, the script will trigger exactly one crash. After all tests have succeeded, the number of crashes will be counted. If the number is one, the test has passed. A number of zero means crash detection isn't working on your system. A number of two or larger means that the NISCC test data triggered an undesired crash in NSS, details should be given to NSS developers who should analyze and fix the bug. Details can be found in /home/niscc/run/out/)

After the execution has completed, you should receive an email. If everything looks right, you are ready to run the full "run", or you could automate execution with a cron job.

Edit file /home/niscc/mycrontab for example to run the job every wednesday just before midnight:

 MAILTO="recipient-email-address-for-reports@domain.com"
 59 23 * * wed /home/niscc/run/run

Install the cronjob

 crontab mycrontab

Note: File run_niscc.sh contains a dynamic request to retrieve the version number of the most recent stable version of NSS. This version number (or source tag) will be used to decide which soure code snapshot will be retrieved and tested. As of today, it uses the versions of NSPR and NSS being used by Mozilla Aurora.

TODO:

• In addition to running the tests against "latest stable" we should also test against "latest development snapshot"
• The logfiles created are large. We need some automatic cleanup of /home/nsicc/run/out/archive and /home/nsicc/run/out/TRASH