MozillaWiki

QA/Sync/Sync Server Install EL6

< QA‎ | Sync
Revision as of 22:38, 4 August 2011 by Jbonacci (talk | contribs) (Created page with "= Notes for installing Sync Server on Red Had Enterprise Linux 6 for 64-Bit systems = REF: http://docs.services.mozilla.com/howtos/run-sync.html == General Notes == * The "Build...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Notes for installing Sync Server on Red Had Enterprise Linux 6 for 64-Bit systems

REF: http://docs.services.mozilla.com/howtos/run-sync.html

General Notes

  • The "Building the server" steps can be repeated for each new build you want to test.
  • Once it is running, you can point to it for your own Sync server.
  • It is highly recommended that Python 2.6.6 be installed before proceeding with the Sync install.
  • Python 2.7 appears to be compatible, but the pre-packaged installer assumes 2.6.6.
  • Also, it is highly recommended that the developer tools are installed after installing Python 2.6.6.

Installing the Developer Tools

To get a full development environment on the box, do the following: 

$ curl -O http://python-distribute.org/distribute_setup.py
$ sudo python2.6 distribute_setup.py
$ sudo easy_install-2.6 MoPyTools

Keeping the build logs

Run the "make build" step as follows: 

$ make build 1> buildlog.log 2> builderr.err

Using Sqlite3

If sqlite3 is to be used, the server-full/etc/sync.conf file must be edited to change the default location of the temp.db file. The default location is in /tmp, which will get wiped at each system boot/restart.

Installing Sync Server

Assuming kernal version similar to the following 2.6.32-131.6.1.el6.x86_64 ... x86_64 x86_64 x86_64 GNU/Linux

Pre-Installed Packages

The following packages were already installed by OPs on the sync6 VM:

  • Python 2.6.6
  • Sqlite3 3.6.20

Required Packages

The following packages need to be installed. This should be a one-off, unless updates/upgrades are needed by the Sync Server code:

  • Mercurial
  • MySQL Server, Devel, etc.
  • Make
  • Virtualenv
  • SetupTools
  • Python virtualenv
  • Python devel
  • Gcc

Installations

  • Install Mercurial 1.4
$ sudo yum install mercurial
  • Verify the installation:
$ hg --version
  • Install MySQL Server 14.14
$ sudo yum install mysql-server
  • Verify the installation:
$ mysql --version
  • Install Gnu Make 3.81
$ sudo yum install make
  • Verify the installation:
$ make -v

Note: For the more manual installation steps, using /usr/local/src as a working directory. Also, borrowed heavily from the following site: http://binarysushi.com/blog/2009/aug/19/CentOS-5-3-python-2-5-virtualevn-mod-wsgi-and-mod-rpaf

  • Install SetupTools
$ cd /usr/local/src
$ sudo wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz
$ sudo tar xzfv setuptools-0.6c11.tar.gz
$ cd setuptools-0.6c11
$ sudo python2.6 setup.py install
  • Install Python virtualenv-1.6.4:
$ cd /usr/local/src
$ sudo wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.6.4.tar.gz
$ sudo tar xvfz virtualenv-1.6.4.tar.gz
$ cd virtualenv-1.6.4
$ sudo python2.6 setup.py install
  • Install Python Devel for Python 2.6.6
$ sudo yum install python-devel-2.6.6-20.el6.x86_64

Note: Need this because we are running 2.6.6 and the VM is x86_64. And, this is required for building the Sync Server and for installing MySQL-Python.

  • Next, install GCC, which is required for MySQL-Python
$ cd /usr/local/src
$ sudo yum install gcc
  • Verify the installation:
$ which gcc
returns the following
/usr/bin/gcc
$ gcc --version
returns the following
gcc (GCC) 4.4.5 20110214 (Red Hat 4.4.5-6)
  • Next, start up MySQL
sudo /etc/init.d/mysqld start
  • Next, configure MySQL with a password and a test DB for Sync.
mysql -u root -p
There may be no password (or a "blank" password)
  • Two commands seems to work to change the password:
mysql> UPDATE mysql.user set Password=PASSWORD('mozroot') WHERE User='root';
mysql> FLUSH PRIVILEGES
  • Create a test database for use with SyncServer
mysql> CREATE DATABASE syncdb;
mysql> \quit

Note: mysql-server install usually prints several lines that show how to do initial setup, so those steps could also be followed instead of those listed above.

  • Next, clone and build the sync server

For this box, we need a Sync Server setup in a more public place. Therefore, going into this open folder for now: /opt/sync 

$ hg clone https://hg.mozilla.org/services/server-full
$ cd server-full
$ make build
  • Next, install mysql-devel (which is a prereq for MySQL-Python)
$ sudo yum install mysql-devel
  • Next, install MySQL-Python
$ cd /opt/sync/server-full
$ bin/easy_install MySQL-Python
  • Next, configure server-full/etc/sync.conf to use MySQL
Change the two "sqluri" lines to match the following (or similar):
sqluri = mysql://root:mozroot@localhost/syncdb
  • Next, configure server-full/etc/sync.conf to use this server as the "fallback":
fallback_node = http://sync6.mtv1.dev.svc.mozilla.com:5000/

Note: IP and port combination for this VM and the Sync Server is the following: 10.250.64.147:5000

  • Now, start up the Sync Server, try the following from /opt/sync/server-full:
$ nohup bin/paster serve development.ini 1>> syncserver.log 2>>syncserver.log &
  • This will run the Sync Server in nohup mode and redirect STDOUT and STDERR to the file syncserver.log.
  • The Sync Server would have to be restarted in case the VM is restarted.

This could be updated by running a startup cron to verify MySQL and Sync Server are restarted properly.

  • Quick testing the Sync Server from any location that can reach the IP (from some ideas Tarek had):
$ curl http://sync6.mtv1.dev.svc.mozilla.com:5000/

This should return a 404 error if it is working correctly

  • From the browser:
$ http://sync6.mtv1.dev.svc.mozilla.com:5000/user/1.0/a

This should return a '0' if it is working correctly

  • Both of these should generate Sync Server INFO messages to STDOUT (from the terminal running Sync Server).
  • Now, you should be able to set up accounts that can sync to the following custom host:
http://sync6.mtv1.dev.svc.mozilla.com:5000/


Rebuilding and Deploying the Sync Server

  • Sync server install is here: /opt/sync/server-full
  • Log is here: /opt/sync/server-full/syncserver.log
  • Kill the Sync Server process
$ kill -9 <PID>

(Leave MySQL running)

  • Save the Sync Server directory and its contents

(this in case we want to keep various snapshots) 

$ cd /opt/sync
$ mv server-full server-full-<DATE>

(example: mv server-full server-full-08012011)

  • Repeat the build and deploy steps from above
  • Make the necessary changes in the server-full/etc/sync.conf file:
  • Change the sqluri and fallback_node
  • Restart the server using the "nohup" command from above
  • Test the service
  • Send out email to services-qa once the service is up and running again

For more advanced configurations, including web server support, please see the following documentation:

For a more technical overview of Sync Server and an introduction to Mozilla Services, please see the following documentation:

Installing/Configuring a Web Server for Sync

TBD: We need to add instructions here for connecting the Sync Server to nginx/gunicorn.

Sync6 appears to have all the prerequisites for doing this work. The following processes are running on sync6: 

memcached -d -p 11211 -u nobody -m 16834 -c 8192 -P /var/run/memcached/memcached.pid
supervise gunicorn-syncstorage
supervise log
supervise nginx
supervise log
nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf -g daemon off;
nginx: worker process                                  
/usr/bin/ruby /usr/sbin/puppetd
nrpe -c /etc/nagios/nrpe.cfg -d
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Sync/Sync_Server_Install_EL6&oldid=336419"