QA/BrowserID/BrowserID Basic Install

From MozillaWiki
Jump to navigation Jump to search

Back To ==> Mozilla BT Services

Instructions for doing a basic install of BrowserID on Mac or Linux

Latest information:

  • Original installs were done with node v0.4.x
  • New installs are being done with node v0.6.x
  • We are moving to Node v0.6.9 for all environments
  • BrowserID now has a requirement for expat-devel library
    • sudo yum -y install expat-devel for CentOS/Fedora/RHEL
    • similar for Ubuntu (apt-get install)

Required Packages

  • XCode (latest for current OS version)
  • gcc
  • g++
  • make
  • openssl
  • node
  • npm
  • git
  • gmp-devel
  • gettext and msgfmt for locale installs and testing
  • svn for locale installs
  • phantomjs for testing
  • mysql for testing

Note: Mac OS should already have the first four, so you should only need to install node, npm, git, etc. before installing BrowserID.

Note 2: With the latest CentOS 6 used for the Linux portion of this document, the following packages were already installed: gcc, g++, make, openssl, git So, if your version/flavor of Linux does not have these installed by default, proceed with installations typical for your Linux.

Installing Node.js and NPM

  • Use the very latest node/npm install to maximize compatibility with gmp and BrowserID:
For example: node v0.6.5 with npm 1.1.0-alpha-6
  • Node can be installed from here:
http://nodejs.org/dist/
Example: http://nodejs.org/dist/v0.6.5/node-v0.6.5.tar.gz
unzip/untar the files
Follow the README.md file to build/install node v0.6.5 (as root):
$ cd node-v0.6.5
$ ./configure --prefix=/usr
$ make
$ sudo make install
  • Verify the installation
$ which node
/usr/bin/node
$ node --version
v0.6.5
$ npm --version
1.1.0-alpha-6

Installing Git on Mac

  • The Mac Git application is available here
http://code.google.com/p/git-osx-installer/downloads/list
  • Select the latest for your version of Mac OS
    • Example: git-1.7.7-intel-universal-snow-leopard.dmg
    • (Note: this should also work for Lion)
  • IMPORTANT: Follow the Readme in the DMG file to complete the installation.
  • Make sure to run the .sh file also, to get your PATH set correctly.
  • For example, I did the following
    • Created my own .sh file as "root"
    • Copied in the contents of the Git application .sh file
    • Saved my file then changed the mode to 777
    • Ran (executed) my .sh file
  • However you do this, remember to sign out and sign back in again to get an updated PATH
  • Check updated PATH env with "set" command
$ set | grep PATH
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin:/usr/X11/bin:/opt/local/bin
  • Verify the Git application and version
$ which git
/usr/local/git/bin/git (for example)

$ git --version
git version 1.7.6.1 (for example)

Installing gmp on Mac

  • There are compatibility issues with using gmp and npm on Mac.
  • The recommendation is to use an older gmp package with the very latest node/npm bundle (v0.6.5).
  • This combination for example, has presented some problems:
    • node v.0.6.3 install that includes npm 1.1.0-alpha-2
  • Using a node Mac package is NOT recommended.
  • Install an older version (stable version) of gmp-devel, for example: gmp-4.3.1.tar.gz
    • (do NOT use gmp-5.0.2)
  • Example site:
ftp://ftp.gnu.org/gnu/gmp/gmp-4.3.1.tar.gz
  • Go through all four installation steps to ensure a decent gmp install
./configure
make
make check
sudo make install
  • The libraries will get installed here: /usr/local/lib

Installing phantomjs on Mac

Grab the latest phantomjs package from here: 
http://code.google.com/p/phantomjs/downloads/list
Example: 
http://phantomjs.googlecode.com/files/phantomjs-1.5.0-macosx-static.zip

Unzip the file
This is a binary package, so no further installation is needed

Create a link from /usr/local/bin to the phantomjs directory
Example:
sudo ln -s /path/to/download/phantomjs-1.5.0/bin/phantomjs /usr/local/bin/phantomjs
Make sure /usr/local/bin is in your path

Installing gmp/gmp-devel on Linux

  • There are compatibility issues with using gmp and npm on Linux, also.
  • The recommendation is to use an older gmp package with the very latest node/npm bundle (v0.6.5).
  • This combination for example, has presented some problems:
    • node v.0.6.3 install that includes npm 1.1.0-alpha-2
  • Install an older version (stable version) of gmp-devel, for example: gmp-4.3.1-7
  • Example site:
ftp://ftp.gnu.org/gnu/gmp/gmp-4.3.1.tar.gz
  • Go through all four installation steps to ensure a decent gmp install
./configure
make
make check
sudo make install
  • Or, use 'yum install' or 'apt-get install' to download and install gmp and/or gmp-devel
  • Example:
$ sudo yum -y install gmp
$ sudo yum -y install gmp-devel
  • Ubuntu 11 notes
    • The gmp install also requires m4 package install: sudo apt-get install m4

Installing phantomjs on Linux

Install the latest version of phantomjs
Example:
wget http://phantomjs.googlecode.com/files/phantomjs-1.5.0-linux-x86_64-dynamic.tar.gz

gunzip and tar xvf the file

Since this is a binary package, just create a link to the executable:
ln -s /path/to/download/bin/phantomjs /usr/local/bin/phantomjs
Make sure /usr/local/bin is in your path

Installing BrowserID

  • From your home directory, run the following
$ git clone git://github.com/mozilla/browserid

$ cd browserid
  • (optional) At this point, you can also choose a specific branch to install:
$ git branch
gives you a list of available branches
$ git branch -a
gives you a more complete list including local and remote branches
$ git checkout "branch"
will switch to the specific "branch"

Note 1: If you do not specifically choose a branch, then, by default, the current Dev branch is installed.
Note 2: For specific train branches, specify them as follows:

$ git checkout train-<DATE>
not like this:
$ git checkout remotes/origin/train-<DATE>
where <DATE> represents the datestamp of the current train/build

$ npm install

Installing BrowserID from RPMs

  • Content to be added soon-ish...

Installing BrowserID with support for Locales

  • In order to run localized versions of BrowserID (localized strings), the following apps/libs must first be installed: svn, gettext (which should include xgettext, msgfmt, etc.).
    • Ubuntu: sudo apt-get install svn gettext
    • RHEL/Fedora/CentOS: sudo yum -y install svn gettext
    • Mac: use Ports, Homebrew, or install from here:
ftp://ftp.gnu.org/pub/gnu/gettext/
Make sure that the gettext/msgfmt bin directory is added to your path
  • If XCode is installed, svn should already be available
  • Continue with the installation of BrowserID:
git clone git://github.com/mozilla/browserid
cd browserid
svn co http://svn.mozilla.org/projects/l10n-misc/trunk/browserid/locale
git checkout "branch" (this is an optional step, see the previous section)
npm install
scripts/compile_mo.sh locale
scripts/compress
npm start
  • Verifying locales
  • Check the contents of this file: browserid/config/l10n-all.json
  • or check the locale listing in the browserid/locale directory

Running BrowserID

  • Run the following command to start up the BrowserID services
$ npm start
  • If this starts up correctly, you will see a number of start up messages and perhaps one debug/error combination:
browserid (2379): debug: opening JSON database: /<YOUR INSTALL DIR>/browserid/var/authdb.json
browserid (2379): error: Cannot read database from /<YOUR INSTALL DIR>/browserid/var/authdb.json
  • This is OK, since there may be no current DB for BrowserID and one has to be made on first use.
  • You should also see that the following services have started:
verifier running on http://127.0.0.1:10000
RP/client running on http://127.0.0.1:10001
browserid server at http://127.0.0.1:10002 (router)
keysigner running on http://127.0.0.1:10003
dbwriter running on http://127.0.0.1:10004
primary running on http://127.0.0.1:10005
proxy running on http://127.0.0.1:10006
browserid running on http://127.0.0.1:10007
nothing for 10008, 10009
static running on http://127.0.0.1:10010
  • Finally, you will also notice that your prompt did not come back. While the servers are running, all messages will go to STDOUT/STDERR as well as the following logs
/<YOUR INSTALL DIR>/browserid/var/log/browserid.log
/<YOUR INSTALL DIR>/browserid/var/log/compress.log
/<YOUR INSTALL DIR>/browserid/var/log/compress-worker.log
/<YOUR INSTALL DIR>/browserid/var/log/dbwriter.log
/<YOUR INSTALL DIR>/browserid/var/log/keysigner-compute.log
/<YOUR INSTALL DIR>/browserid/var/log/keysigner.log
/<YOUR INSTALL DIR>/browserid/var/log/proxy.log
/<YOUR INSTALL DIR>/browserid/var/log/router.log
/<YOUR INSTALL DIR>/browserid/var/log/router-metrics.json
/<YOUR INSTALL DIR>/browserid/var/log/run_locally.log
/<YOUR INSTALL DIR>/browserid/var/log/static.log
/<YOUR INSTALL DIR>/browserid/var/log/test_db_connectivity.log
/<YOUR INSTALL DIR>/browserid/var/log/test_frontend.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier-compute.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier.log
/<YOUR INSTALL DIR>/browserid/var/log/verifier-metrics.json
/<YOUR INSTALL DIR>/browserid/var/log/vows.log
  • If you do not want the services to write to STDOUT/STDERR, then just redirect both when starting npm
nohup npm start 1>> MYLog.log 2>>MYLog.log &
or with an IP
IP_ADDRESS=<YOUR LOCAL IP> nohup npm start 1>> MYLog.log 2>>MYLog.log &

Using BrowserID

  • To create an account and sign in and out, use the client address and port
http://127.0.0.1:10001
  • To use the main server and Account Manager, use the server address and port:
http://127.0.0.1:10002
  • Note: the keysigner, verifier, and dbwriter are active behind the scenes, so no user interaction is needed here.
  • You can use this "client" and this server as you would the Beta or Production BrowserID environments. The only missing element is the "content" of the client site, which, in this case, is basically limited to creation, signing in, adding emails, signing out, etc.


Creating and Using BrowserID Accounts

  • You can create an account on the client (http://127.0.0.1:10001/) or onthe server (http://127.0.0.1:10001/).
  • The email verification does not actually get sent because there is nothing hooked up to do this.
  • The email verification string will appear in STDOUT (or in the log if you redirected STDOUT/STDERR):
  • Copy and paste that verification string into your browser and you will be verified.
  • Now, you should be able to log into either site and use that account.

Client Site Information

  • Client site: http://127.0.0.1:10001/
  • This site now allows for four types of RP activity:
    • Silent mode
    • Allow persistent sign-in
    • Require a specific email
    • Get an assertion
  • To see what is actually happening, track the events and messages in the console log (STDOUT/STDERR)

Primary Information

  • Example Primary: http://127.0.0.1:10005
  • This site currently just does simple sign in and sign out of an example primary
  • Checking the log will show the GETs used
  • Here is an example:
example_primary (2027): GET / 304 1ms
example_primary (2027): GET /jquery.js 304 1ms
example_primary (2027): GET /api/whoami 200 2ms
example_primary (2027): GET /api/login?user=John.Doe%40MyMail.com 200 1ms
example_primary (2027): GET /api/whoami 200 1ms
example_primary (2027): GET /api/logout 200 2ms
example_primary (2027): GET /api/whoami 200 1ms

Logging

  • The BrowserID logs exist in your browserid/var/log directory
browserid.log
verifier.log
verifier-metrics.json
verifier-compute.log
router.log
static.log
router-metrics.json
dbwriter.log
keysigner.log
keysigner-compute.log
databases: n/a
zeus: various logs
  • Note: router-metrics.json replaces browserid-metrics.json
  • Note: These logs now have time/date stamps.

BrowserID Processes

  • If you search for "node" processes, you will see that seven are actively running:
node ./scripts/run_locally.js
node /<YOUR INSTALL DIR>/browserid/bin/verifier
node /<YOUR INSTALL DIR>/browserid/bin/keysigner
node /<YOUR INSTALL DIR>/browserid/bin/dbwriter
node /<YOUR INSTALL DIR>/browserid/scripts/serve_example.js
node /<YOUR INSTALL DIR>/browserid/scripts/serve_example_primary.js
node /<YOUR INSTALL DIR>/browserid/bin/proxy
node /<YOUR INSTALL DIR>/browserid/bin/browserid
node /<YOUR INSTALL DIR>/browserid/bin/router
node /<YOUR INSTALL DIR>/browserid/bin/node bin/static

Quitting BrowserID

  • Hitting CTRL-c from the terminal window where you ran "npm start" should kill all "node" processes.
  • If you are running from the background (for example to trap all STDOUT and STDERR messages), just look for and kill the following process:
node /usr/local/bin/npm start
  • If either method does not clean up after itself, you may need to manually kill the remaining node processes:
$ ps aux | grep node

Running Unit Tests

  • The front-end, back-end, and "headless" unit tests can be run from a local install of BrowserID.
  • The back-end and "headless" unit tests have now been combined and can be run with the single command "npm test"
  • Also the updated back-end unit tests now use phantomjs, so following the instructions in the previous sections for installing it and setting the path.
  • You will need to know the local IP of the test system for the front-end unit tests.
Mac/Linux Inet Addr info is from 'ifconfig'
Windows IP Addr info is from 'ipconfig'


Front-End Unit Tests in the Browser

  • Given the inet (local IP) of the system, run the following command from the browserid folder:
$ IP_ADDRESS=<YOUR LOCAL IP> npm start
Example: $ IP_ADDRESS=50.50.50.50 npm start
  • Then from the browser, access the front-end unit test as follows:
http://<YOUR LOCAL IP>:10002/test/
Example: http://50.50.50.50:10002/test/
(where 10002 is port value for local BrowserID server/PIA/IP)
  • You can use this IP and the running BrowserID instance to execute the unit tests in any browser on your desktop, a VM, or other machines that can see the desktop/IP.
  • Note: you can no longer run the front-end unit tests against Stage or Prod. You can still run them against Dev
https://dev.diresworb.org/test/
  • Unit test location in code (and GitHub)
browserid/resources/static/test/qunit

Back-End Unit Tests with PhantomJS and MySQL Support

  • The back-end unit tests (which now include the "headless" unit tests) can be run locally as follows, from the browserid folder:
$ npm test
(The services do not need to be running, the test app takes care of everything)
  • And, for mysql support without creating the necessary test user and test db, just use the following environment variables:
$ NODE_ENV=test_mysql MYSQL_USER=browserid MYSQL_PASSWORD=browserid npm test
  • Make sure that phantomjs is installed and the path has been set before running these (see notes in the previous sections)
  • Make sure that mysql is installed and running with a test user and test db already created (see the next section)
  • Unit test location in code (and GitHub)
browserid/tests
  • Note: the Back-End Unit Tests can not be run on Stage or Prod due to the focused server-style RPM install and the limited permissions for QA.

Installing MySQL and MySQL Server

  • MySQL is a prerequisite for running the unit tests and load_gen on a local install.
Use any common method for installing mysql and mysql-server packages to the local environment: Mac or Linux
Example: sudo yum install mysql mysql-server
Start up mysql and set to load/restart automatically
Example: sudo /etc/init.d/mysqld start; sudo chkconfig --levels 235 mysqld on
As "root", create a "browserid" user with full privileges and no password
mysql -u root -p
CREATE DATABASE test_mysql;
GRANT ALL PRIVILEGES ON *.* TO 'browserid'@'localhost';
UPDATE mysql.user SET Password=PASSWORD('browserid') WHERE User = 'browserid';
FLUSH PRIVILEGES;
\quit
  • Alternatively, environment variables can be used to run the unit tests with mysql support (assuming mysql is installed and running):
NODE_ENV=test_mysql MYSQL_USER=browserid MYSQL_PASSWORD=browserid npm test

The load_gen Application

  • load_gen flags/parameters
a: activities (specified in an CSV)
h: help
l: list (list available activities and exit)
m: max active users (default = 10000)
o: omit-static (default = false)
s: server (base URL)
v: verifier (base URL) (default = server/verify)
u: user range
  • Standard Usage
cd ~/browserid/
bin/load_gen -u <user range> -m <max users> -o -s http://127.0.0.1:10002
  • Note 1: the other flags/parameters are optional
  • Note 2: for every 1000 ADU (-m value), you need 1 user (-u value)

Running a Load Test on the localhost

  • Start up BrowserID with some required environment variables
CREATE_TEST_USERS=2000 BROWSERID_FAKE_VERIFICATION=1 NODE_ENV=test_mysql npm start
  • This will start BrowserID and create a test set of 2000 users and make use of a test mysql database.
  • In another terminal start up the load_gen application
Example: bin/load_gen -u 1/20 -m 20000 -o -s http://127.0.0.1:10002
  • Redirecting STDOUT and STDERR
Example for BrowserID:
CREATE_TEST_USERS=2000 BROWSERID_FAKE_VERIFICATION=1 NODE_ENV=test_mysql nohup npm start 1>> NPM1.log 2>>NPM1.log &
Example for load_gen:
nohup bin/load_gen -u 1/20 -m 20000 -o -s http://127.0.0.1:10002 1>> LoadTest1.log 2>>LoadTest1.log &
  • Understanding the load_gen output
    • The output from load_gen to STDOUT is a row of four columns, one row per second:
Example:
503783.28       508388.70       500331.03       52 R, 232 S (2sp 0rs 0al 1rh 49sn 0iy 0cs)
    • The first column (503783.28) is the average simulated users over the last second
    • The second column (508388.70) is the average simulated users over the last 5 seconds
    • The third column (500331.03) is the average simulated users over the last 60 seconds
    • The fourth column is made up of two values and another set of 7 values in parentheses
      • R value (52 R) is the number of simultaneous "activities" that are currently running
      • S value (232 S) is the number of new "activities" started this second
      • 7 values are the number of activities running right now by type
  • Error tracking
    • Standard HTTP messages and BrowserID messages (info, debug, warn, error) can be tracked from the STDOUT of "npm start"
    • Specific types of BrowserID activity and errors can be tracked from the STDOUT of "load_gen"
      • Examples:
      • error: can't certify key: non-200 status: 400NaNBad Request: that email does not belong to you
      • error: failed to authenticate: non-success response 200 - {"success":false,"reason":"no such user"}
  • Performance Tracking
    • TOP can be used (or even Activity Monitor on Mac) to track CPU, memory, process time, etc.


Back To ==> Mozilla BT Services

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/BrowserID/BrowserID_Basic_Install&oldid=456324"