QA/BrowserID/BrowserID Basic Install

From MozillaWiki
< QA‎ | BrowserID
Jump to: navigation, search

Back To ==> Mozilla BT Services

Instructions for doing a basic install of Persona/BrowserID on Mac or Linux

Latest information:

  • Original installs were done with node v0.4.x
  • New installs are being done with node v0.6.x
  • As of March 2013, we are using v0.8.17
  • Persona/BrowserID now has a requirement for expat-devel library
    • sudo yum -y install expat-devel for CentOS/Fedora/RHEL
    • similar for Ubuntu (apt-get install)

Required Packages

  • XCode (latest for current OS version)
  • gcc
  • g++
  • make
  • openssl
  • node
  • npm
  • git
  • gmp-devel
  • gettext and msgfmt for locale installs and testing
  • svn for locale installs
  • phantomjs for testing
  • mysql for testing
  • Mac Notes:
    • Mac OS should already have the first four, so you should only need to install node, npm, git, etc. before installing Persona/BrowserID.
    • With the latest versions of Mac OS and XCode, the command line tools are not downloaded/installed by default. You must explicitly download them through Xcode > Preferences > Downloads.
  • Linux Notes:
    • With the latest CentOS 6 used for the Linux portion of this document, the following packages were already installed: gcc, g++, make, openssl, git
    • So, if your version/flavor of Linux does not have these installed by default, proceed with installations typical for your Linux.

Installing Node.js and NPM

  • Use the very latest node/npm install to maximize compatibility with gmp and Persona/BrowserID:
For example: node v0.6.5 with npm 1.1.0-alpha-6
  • Node can be installed from here:
https://nodejs.org/dist/
Example: https://nodejs.org/dist/v0.6.5/node-v0.6.5.tar.gz
unzip/untar the files
Follow the README.md file to build/install node v0.6.5 (as root):
$ cd node-v0.6.5
$ ./configure --prefix=/usr
$ make
$ sudo make install
  • Verify the installation
$ which node
/usr/bin/node
$ node --version
v0.6.5
$ npm --version
1.1.0-alpha-6

Installing Git on Mac

  • The Mac Git application is available here
https://code.google.com/p/git-osx-installer/downloads/list
  • Select the latest for your version of Mac OS
    • Example: git-1.7.7-intel-universal-snow-leopard.dmg
    • (Note: this should also work for Lion)
  • IMPORTANT: Follow the Readme in the DMG file to complete the installation.
  • Make sure to run the .sh file also, to get your PATH set correctly.
  • For example, I did the following
    • Created my own .sh file as "root"
    • Copied in the contents of the Git application .sh file
    • Saved my file then changed the mode to 777
    • Ran (executed) my .sh file
  • However you do this, remember to sign out and sign back in again to get an updated PATH
  • Check updated PATH env with "set" command
$ set | grep PATH
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/usr/X11/bin:/opt/local/bin
  • Verify the Git application and version
$ which git
/usr/local/git/bin/git (for example)

$ git --version
git version 1.7.6.1 (for example)

Installing gmp on Mac

  • NOTE: It should be possible to install the latest libgmp package, which can be found here: https://gmplib.org/download/ (Using a node Mac package is NOT recommended.)
  • (OLD INFORMATION) There were compatibility issues with using gmp and npm on Mac. The recommendation was to use an older gmp package with the very latest node/npm bundle.
  • Go through all four installation steps to ensure a decent gmp install
./configure
make
make check
sudo make install
  • The libraries will get installed here: /usr/local/lib

Installing phantomjs on Mac

Grab the latest phantomjs package from here: 
https://code.google.com/p/phantomjs/downloads/list
Example: 
https://phantomjs.googlecode.com/files/phantomjs-1.5.0-macosx-static.zip

Unzip the file
This is a binary package, so no further installation is needed

Create a link from /usr/local/bin to the phantomjs directory
Example:
sudo ln -s /path/to/download/phantomjs-1.5.0/bin/phantomjs /usr/local/bin/phantomjs
Make sure /usr/local/bin is in your path

Installing gmp/gmp-devel on Linux

  • NOTE: It should be possible to install the latest libgmp package, which can be found here: https://gmplib.org/#DOWNLOAD
  • (OLD INFORMATION) There were compatibility issues with using gmp and npm on Linux, also. The recommendation was to use an older gmp package with the very latest node/npm bundle.
  • Go through all four installation steps to ensure a decent gmp install
./configure
make
make check
sudo make install
  • Or, use 'yum install' or 'apt-get install' to download and install gmp and/or gmp-devel
  • Example:
$ sudo yum -y install gmp
$ sudo yum -y install gmp-devel
  • Ubuntu 11 notes
    • The gmp install also requires m4 package install: sudo apt-get install m4

Installing phantomjs on Linux

Install the latest version of phantomjs
Example:
wget https://phantomjs.googlecode.com/files/phantomjs-1.5.0-linux-x86_64-dynamic.tar.gz

gunzip and tar xvf the file

Since this is a binary package, just create a link to the executable:
ln -s /path/to/download/bin/phantomjs /usr/local/bin/phantomjs
Make sure /usr/local/bin is in your path

Installing Persona/BrowserID

  • From your home directory, run the following
$ git clone git://github.com/mozilla/browserid

$ cd browserid
  • (optional) At this point, you can also choose a specific branch to install:
$ git branch
gives you a list of available branches
$ git branch -a
gives you a more complete list including local and remote branches
$ git checkout "branch"
will switch to the specific "branch"

Note 1: If you do not specifically choose a branch, then, by default, the current Dev branch is installed.
Note 2: For specific train branches, specify them as follows:

$ git checkout train-<DATE>
not like this:
$ git checkout remotes/origin/train-<DATE>
where <DATE> represents the datestamp of the current train/build

$ npm install

Installing Persona/BrowserID from RPMs

  • Content to be added soon-ish...

Installing Persona/BrowserID with support for Locales

https://ftp.gnu.org/pub/gnu/gettext/
Make sure that the gettext/msgfmt bin directory is added to your path
  • If XCode is installed, svn should already be available
  • Continue with the installation of Persona/BrowserID:
git clone git://github.com/mozilla/browserid
cd browserid
svn co http://svn.mozilla.org/projects/l10n-misc/trunk/browserid/locale
git checkout "branch" (this is an optional step, see the previous section)
npm install
scripts/compile_mo.sh locale
scripts/compress
npm start
  • Verifying locales
  • Check the contents of this file: browserid/config/l10n-all.json
  • or check the locale listing in the browserid/locale directory

Running Persona/BrowserID

  • Run the following command to start up the Persona/BrowserID services
$ npm start
  • If this starts up correctly, you will see a number of start up messages and perhaps one debug/error combination:
browserid (2379): debug: opening JSON database: /<YOUR INSTALL DIR>/browserid/var/authdb.json
browserid (2379): error: Cannot read database from /<YOUR INSTALL DIR>/browserid/var/authdb.json
  • This is OK, since there may be no current DB for Persona/BrowserID and one has to be made on first use.
  • You should also see that the following services have started:
verifier running on http://127.0.0.1:10000
RP/client running on http://127.0.0.1:10001
browserid server at http://127.0.0.1:10002 (router)
keysigner running on http://127.0.0.1:10003
dbwriter running on http://127.0.0.1:10004
primary running on http://127.0.0.1:10005
proxy running on http://127.0.0.1:10006
browserid running on http://127.0.0.1:10007
nothing for 10008, 10009
static running on http://127.0.0.1:10010
Delegated Primary running on http://127.0.0.1:10011
  • Finally, you will also notice that your prompt did not come back. While the servers are running, all messages will go to STDOUT/STDERR as well as the following logs
/<YOUR INSTALL DIR>/browserid/var/log/browserid.log
/<YOUR INSTALL DIR>/browserid/var/log/compress.log
/<YOUR INSTALL DIR>/browserid/var/log/compress-worker.log
/<YOUR INSTALL DIR>/browserid/var/log/dbwriter.log
/<YOUR INSTALL DIR>/browserid/var/log/keysigner-compute.log
/<YOUR INSTALL DIR>/browserid/var/log/keysigner.log
/<YOUR INSTALL DIR>/browserid/var/log/proxy.log
/<YOUR INSTALL DIR>/browserid/var/log/router.log
/<YOUR INSTALL DIR>/browserid/var/log/router-metrics.json
/<YOUR INSTALL DIR>/browserid/var/log/run_locally.log
/<YOUR INSTALL DIR>/browserid/var/log/static.log
/<YOUR INSTALL DIR>/browserid/var/log/test_db_connectivity.log
/<YOUR INSTALL DIR>/browserid/var/log/test_frontend.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier-compute.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier-metrics.json
/<YOUR INSTALL DIR>/browserid/var/log/vows.log
  • If you do not want the services to write to STDOUT/STDERR, then just redirect both when starting npm
nohup npm start 1>> MYLog.log 2>>MYLog.log &
or with an IP
IP_ADDRESS=<YOUR LOCAL IP> nohup npm start 1>> MYLog.log 2>>MYLog.log &

Using Persona/BrowserID

  • To create an account and sign in and out, use the client address and port
http://127.0.0.1:10001
  • To use the main server and Account Manager, use the server address and port:
http://127.0.0.1:10002
  • Note: the keysigner, verifier, and dbwriter are active behind the scenes, so no user interaction is needed here.
  • You can use this "client" and this server as you would the Beta or Production Persona/BrowserID environments. The only missing element is the "content" of the client site, which, in this case, is basically limited to creation, signing in, adding emails, signing out, etc.


Creating and Using Persona/BrowserID Accounts

  • You can create an account on the client (http://127.0.0.1:10001/) or onthe server (http://127.0.0.1:10001/).
  • The email verification does not actually get sent because there is nothing hooked up to do this.
  • The email verification string will appear in STDOUT (or in the log if you redirected STDOUT/STDERR):
  • Copy and paste that verification string into your browser and you will be verified.
  • Now, you should be able to log into either site and use that account.

Client Site Information

  • Client site: http://127.0.0.1:10001/
  • This site now allows the following types of RP activity (assertions):
    • Supply a privacy policy
    • Supply a ToS
    • Supply Site Name
    • Supply Site Logo
    • Supply returnTo
  • To see what is actually happening, track the events and messages in the console log (STDOUT/STDERR)

Primary Information

  • Example Primary: http://127.0.0.1:10005
  • This site currently just does simple sign in and sign out of an example primary
  • Checking the log will show the GETs used
  • Here is an example:
example_primary (2027): GET / 304 1ms
example_primary (2027): GET /jquery.js 304 1ms
example_primary (2027): GET /api/whoami 200 2ms
example_primary (2027): GET /api/login?user=John.Doe%40MyMail.com 200 1ms
example_primary (2027): GET /api/whoami 200 1ms
example_primary (2027): GET /api/logout 200 2ms
example_primary (2027): GET /api/whoami 200 1ms

Logging

  • The Persona/BrowserID logs exist in your browserid/var/log directory
browserid.log
compress.log
compress-worker.log
dbwriter.log
keysigner-compute.log
keysigner.log
proxy.log
router.log
router-metrics.json
run_locally.log
static.log
test_db_connectivity.log
test_frontend.log
verifier-compute.log
verifier.log
verifier-metrics.json
vows.log
databases: n/a
zeus: various logs
  • Note: router-metrics.json replaces browserid-metrics.json
  • Note: These logs now have time/date stamps.

Persona/BrowserID Processes

  • If you search for "node" processes, you will see that seven are actively running:
node ./scripts/run_locally.js
node /<YOUR INSTALL DIR>/browserid/bin/verifier
node /<YOUR INSTALL DIR>/browserid/bin/keysigner
node /<YOUR INSTALL DIR>/browserid/bin/dbwriter
node /<YOUR INSTALL DIR>/browserid/scripts/serve_example.js
node /<YOUR INSTALL DIR>/browserid/scripts/serve_example_primary.js
node /<YOUR INSTALL DIR>/browserid/bin/proxy
node /<YOUR INSTALL DIR>/browserid/bin/browserid
node /<YOUR INSTALL DIR>/browserid/bin/router
node /<YOUR INSTALL DIR>/browserid/bin/static

Quitting Persona/BrowserID

  • Hitting CTRL-c from the terminal window where you ran "npm start" should kill all "node" processes.
  • If you are running from the background (for example to trap all STDOUT and STDERR messages), just look for and kill the following process:
node /usr/local/bin/npm start
  • If either method does not clean up after itself, you may need to manually kill the remaining node processes:
$ ps aux | grep node

Installing MySQL and MySQL Server

  • MySQL is a prerequisite for running the unit tests and load_gen on a local install.
Use any common method for installing mysql and mysql-server packages to the local environment: Mac or Linux
Example: sudo yum install mysql mysql-server
Start up mysql and set to load/restart automatically
Example: sudo /etc/init.d/mysqld start; sudo chkconfig --levels 235 mysqld on
As "root", create a "browserid" user with full privileges and no password
mysql -u root -p
CREATE DATABASE test_mysql;
GRANT ALL PRIVILEGES ON *.* TO 'browserid'@'localhost';
UPDATE mysql.user SET Password=PASSWORD('browserid') WHERE User = 'browserid';
FLUSH PRIVILEGES;
\quit
  • Alternatively, environment variables can be used to run the unit tests with mysql support (assuming mysql is installed and running):
NODE_ENV=test_mysql MYSQL_USER=browserid MYSQL_PASSWORD=browserid npm test

Running Unit Tests

  • The front-end, back-end, and "headless" unit tests can be run from a local install of Persona/BrowserID.
  • The back-end and "headless" unit tests have now been combined and can be run with the single command "npm test"
  • Also the updated back-end unit tests now use phantomjs, so following the instructions in the previous sections for installing it and setting the path.
  • You will need to know the local IP of the test system for the front-end unit tests.
Mac/Linux Inet Addr info is from 'ifconfig'
Windows IP Addr info is from 'ipconfig'


Front-End Unit Tests in the Browser

  • Given the inet (local IP) of the system, run the following command from the browserid folder:
$ IP_ADDRESS=<YOUR LOCAL IP> npm start
Example: $ IP_ADDRESS=50.50.50.50 npm start
  • Then from the browser, access the front-end unit test as follows:
http://<YOUR LOCAL IP>:10002/test/
Example: http://50.50.50.50:10002/test/
(where 10002 is port value for local Persona/BrowserID server/PIA/IP)
  • You can use this IP and the running Persona/BrowserID instance to execute the unit tests in any browser on your desktop, a VM, or other machines that can see the desktop/IP.
  • Note: you can no longer run the front-end unit tests against Stage or Prod. You can still run them against Dev
https://dev.diresworb.org/test/
  • Unit test location in code (and GitHub)
browserid/resources/static/test/qunit

Back-End Unit Tests with PhantomJS and MySQL Support

  • The back-end unit tests (which now include the "headless" unit tests) can be run locally as follows, from the browserid folder:
$ npm test
(The services do not need to be running, the test app takes care of everything)
  • And, for mysql support without creating the necessary test user and test db, just use the following environment variables:
$ NODE_ENV=test_mysql MYSQL_USER=browserid MYSQL_PASSWORD=browserid npm test
  • Make sure that phantomjs is installed and the path has been set before running these (see notes in the previous sections)
  • Make sure that mysql is installed and running with a test user and test db already created (see the next section)
  • Unit test location in code (and GitHub)
browserid/tests
  • Note: the Back-End Unit Tests can not be run on Stage or Prod due to the focused server-style RPM install and the limited permissions for QA.

Running the Automation Tests

  • Selenium-style automation tests can be run from any local install of Persona/BrowserID. The tests use a standalone selenium server (downloaded separately)
  • The automation tests can be found in the following directory:
    • browserid/automation-tests/scripts
  • The main script will start up a local instance of FireFox (or whatever browser is chosen) to run the client-based automated tests.
  • Basic instructions
    • Download the standalone Selenium server from the following site
https://selenium.googlecode.com
Example: curl -O https://selenium.googlecode.com/files/selenium-server-standalone-2.31.0.jar
    • In one terminal, run the server
java -jar selenium-server-standalone-2.31.0.jar
    • An example using the chrome driver support
exec java -Dwebdriver.chrome.driver=/<PATH TO DRIVER>/chromedriver 
     -jar /<PATH TO JAR>/selenium-server-standalone-2.301.0.jar
    • Open another terminal to run the automated tests
cd browserid/automation-tests/scripts
./scripts/run-all.js --help
will give you the current list of all automation commands

./scripts/run-all.js --local --list-platforms
will give you the current list of all supported platforms on the local host
(this is a generic list that assumes all/any browser(s) can be run on the local host)
Edit the following file to change/update this list, including adding support for all FireFox channels:
browserid/automation-tests/config/local-platform.js

./scripts/run-all.js --list-platforms
will give you the current list of all supported platforms for running against sauce labs
(this list should not be updated)

By default, the automation tests only run against the FireFox browser (any current version
on any current OS) without the use of a special browser driver.
See the following site for more information on downloading drivers to support other browsers:
https://docs.seleniumhq.org/download/
NOTE: As of Selenium webdriver 2.31, Safari and Opera should have built-in support.
The webdrivers for IE and Google Chrome are still required.

./scripts/run-all.js --list-tests
will give you the current list of all automated tests than can be run singly or together

./scripts/run-all.js --env <other options here>
will allow you to change which environment you test: dev, stage, prod (the default is dev)
  • Examples against local host
One test, one browser, locally:
./scripts/run-all.js --local --platform=firefox --tests change-password-test
./scripts/run-all.js --local --platform=linux --tests change-password-test
All tests, one browser, locally:
./scripts/run-all.js --local --platform=firefox-aurora
All tests, one browser, locally, against stage:
./scripts/run-all.js --local --env stage --platform=firefox-beta
  • Examples against sauce labs
One test, one browser:
./scripts/run-all.js --platform=osx_firefox_19 --tests change-password-test
./scripts/run-all.js --platform=linux_opera_12 --tests change-password-test
All tests, one browser:
./scripts/run-all.js --platform=win7_chrome
All tests, all browsers:
./scripts/run-all.js --platform=all
  • Notes:
    • If you have multiple versions of Firefox installed on your system, you can tweak the local-platform.js file and add each version as an entry (see the comment in the file for an example)
    • The current list of supported platforms for Sauce labs can always be found here: browserid/automation-tests/config/sauce-platforms.js file
    • The local host tests will create a significant amount of information that can be seen in the STDOUT/STDERR of the stand-alone server. Therefore, it might be helpful to redirect both to a log file.
    • Sauce Labs failures will generate an error link similar to the following: https://saucelabs.com/jobs/<Some Test ID>. Clicking on the link will give some detail to the failure(s) generated from the preceding automation test run.
    • The --output xunit flag can also be used to generate a live link for the sauce labs run

The load_gen Application

  • load_gen flags/parameters
 -a, --activities        only run a subset of activities, specified as a CSV list                                                  
 -h, --help              display this usage message                                                                                
 -l, --list              list available activities and exit                                                                        
 -m, --max               maximum active users to simulate (0 == infinite) [default: 10000]
 -o, --omit-static       when enabled, only dynamic WSAPI calls will be simulated, not
                         static resource requests [default: false]
 -i, --interaction-data  sort of like `data_sample_rate` (set to 0 to send nothing) [default: 0.1]
 -s, --server            base URL to browserid server [string]
 -v, --verifier          base URL to verifier service (default is browserid server + '/verify')                                    
 -u, --user-range        rather than creating users, assume a range of users exist #@loadtest.domain 
                         (with password "THE PASSWORD")
  • Standard Usage
cd ~/browserid/
bin/load_gen -u <user range> -m <max users> -o -s http://127.0.0.1:10002
  • Note 1: the other flags/parameters are optional
  • Note 2: for every 1000 ADU (-m value), you need 1 user (-u value)

Running a Load Test on the localhost

  • Start up Persona/BrowserID with some required environment variables
CREATE_TEST_USERS=2000 BROWSERID_FAKE_VERIFICATION=1 NODE_ENV=test_mysql npm start
  • This will start Persona/BrowserID and create a test set of 2000 users and make use of a test mysql database.
  • In another terminal start up the load_gen application
Example: bin/load_gen -u 1/20 -m 20000 -o -s http://127.0.0.1:10002
  • Redirecting STDOUT and STDERR
Example for Persona/BrowserID:
CREATE_TEST_USERS=2000 BROWSERID_FAKE_VERIFICATION=1 NODE_ENV=test_mysql nohup npm start 1>> NPM1.log 2>>NPM1.log &
Example for load_gen:
nohup bin/load_gen -u 1/20 -m 20000 -o -s http://127.0.0.1:10002 1>> LoadTest1.log 2>>LoadTest1.log &
  • Understanding the load_gen output
    • The output from load_gen to STDOUT is a row of four columns, one row per second:
Example:
503783.28       508388.70       500331.03       52 R, 232 S (2sp 0rs 0al 1rh 49sn 0iy 0cs)
    • The first column (503783.28) is the average simulated users over the last second
    • The second column (508388.70) is the average simulated users over the last 5 seconds
    • The third column (500331.03) is the average simulated users over the last 60 seconds
    • The fourth column is made up of two values and another set of 7 values in parentheses
      • R value (52 R) is the number of simultaneous "activities" that are currently running
      • S value (232 S) is the number of new "activities" started this second
      • 7 values are the number of activities running right now by type
  • Error tracking
    • Standard HTTP messages and Persona/BrowserID messages (info, debug, warn, error) can be tracked from the STDOUT of "npm start"
    • Specific types of Persona/BrowserID activity and errors can be tracked from the STDOUT of "load_gen"
      • Examples:
      • error: can't certify key: non-200 status: 400NaNBad Request: that email does not belong to you
      • error: failed to authenticate: non-success response 200 - {"success":false,"reason":"no such user"}
  • Performance Tracking
    • TOP can be used (or even Activity Monitor on Mac) to track CPU, memory, process time, etc.


Back To ==> Mozilla BT Services

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/BrowserID/BrowserID_Basic_Install&oldid=1199301"