QA/Sync/Sync Server Install Fedora: Difference between revisions

From MozillaWiki
< QA‎ | Sync
Jump to navigation Jump to search
(Created page with "= Notes for Installing Sync Server on Fedora 15 = REF: http://docs.services.mozilla.com/howtos/run-sync.html == General Notes == * The "Building the server" steps can be repeate...")
 
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
Back To ==> [[QA/Browser_Technologies/Services | Mozilla BT Services]]
= Notes for Installing Sync Server on Fedora 15 =
= Notes for Installing Sync Server on Fedora 15 =
REF: http://docs.services.mozilla.com/howtos/run-sync.html
REF: http://docs.services.mozilla.com/howtos/run-sync.html
Line 21: Line 24:


== Using Sqlite3 ==
== Using Sqlite3 ==
If sqlite3 is to be used, the sync.conf file must be edited to change the default location of the temp.db file. The default location is in /tmp, which will get wiped at each system boot/restart.
If sqlite3 is to be used, the server-full/etc/sync.conf file must be edited to change the default location of the temp.db file. The default location is in /tmp, which will get wiped at each system boot/restart.


== Installing Sync Server ==
== Installing Sync Server ==
Line 29: Line 32:
The Sync Server can be re-configured to use MySQL instead of Sqlite3.
The Sync Server can be re-configured to use MySQL instead of Sqlite3.


*  
* Install MySQL
 
$ sudo yum install mysql, mysql-devel, mysql-server
 
Note: mysql-python will also have to be installed since Fedora 15 does not pre-install this package.
Quick workaround:
$ cd server_full
Replace sqlite3 usage with MySQL on Fedora.
  $ bin/easy_install MySQL-Python
Note that mysql-python will also have to be installed.
Since Fedora 15 does not seem to have it preinstalled...
Install mysql, mysql-devel, mysql-server:
sudo yum install....
 
Change two lines in the server-full/etc/sync.conf file:
Original:
sqluri = sqlite:////tmp/test.db
Updated:
sqluri = mysql://user:password@server/database
where "user" is the current user on this install, like "mozilla" or "mozroot"
"password" is the mysql password for this user
"server" is just localhost
"database" is a default or new database for use with the Sync server
Steps to set all this up are below...
 
Further steps to transition over to MySQL:
 
1. Start MySQL
sudo /etc/init.d/mysqld start
(will ask for mozroot password, obviously)
Running the following:
Server version: 5.5.13 MySQL Community Server (GPL)
or
mysql Ver 14.14 Distrib 5.5.14, for Linux (i686) using readline 5.1
 
Need the steps or commands here to start the server automatically after system restart...
 
 
2. Ran the following to enter mysql:
mysql -u root -p
There is no password (blank)
 
 
3. Two commands seems to work to change the password:
mysql> UPDATE mysql.user set Password=PASSWORD('mozroot') WHERE User='root';
mysql> FLUSH PRIVILEGES
then
mysql> \quit
 
 
4. Create a dummy database for use with SyncServer
mysql> CREATE DATABASE syncdb;
mysql> \quit
 
 
5. Fixed ../server-full/etc/sync.conf
sqluri = mysql://root:mozroot@localhost/syncdb
(both places in the file)
 
 
6. Final step should be to install mysql-python:
bin/easy_install MySQL-Python
 
 
Server is now running again:
Starting server in PID 3568.
serving on 0.0.0.0:5000 view at http://127.0.0.1:5000
 
Looks good so far, but how to actually use it in a VM?
Multiple instances of FF?
SSH tunneling and port forwarding?
 
Test the server from FF in Fedora. Not much to do here since we do not think we can
reference the server from outside the VM.
What about Syncing across two profiles with same account?
 
Also, how do we restart it now?
Do I need to restart MySQL server?
 
What happens after cloning and build a new version?
Remember to save off the sync.conf!
 
Possible lines for autostarting MySQL server on boot
sudo /etc/init.d/mysqld start ## use restart after update
## OR ##
sudo service mysqld start ## use restart after update
sudo chkconfig --levels 235 mysqld on
 


Fedora profile and account:
* Start MySQL
Email: jbonacci+fedora1@mozilla.com
$ sudo /etc/init.d/mysqld start
Pwd: the usual
(will ask for the root password, if set)
Server: http://127.0.0.1:5000
will show something like the following:
Key: w-tcs7d-gq6nh-3zhna-kwhxg-wrcea
Server version: 5.5.13 MySQL Community Server (GPL)
Key2: 6-pvxnx-caz4i-b8n2s-3vyfq-4rnbq
or
Test Profiles: Preferences > Privacy > Remember History
mysql  Ver 14.14 Distrib 5.5.14, for Linux (i686) using readline 5.1


* Run the following to enter mysql:
$ mysql -u root -p
There may be no password yet (in which case the password field is blank)


* If there is no password, run following to create one
mysql> UPDATE mysql.user set Password=PASSWORD('mozroot') WHERE User='root';
mysql> FLUSH PRIVILEGES
then
mysql> \quit


---------------------------------------------------------------------------------------------------
* While in mysql, create a database for use with SyncServer
mysql> CREATE DATABASE syncdb;
mysql> \quit


Latest clone and build is full of warnings and errors
* Change two lines in the server-full/etc/sync.conf file
See /home/mozroot/server-full
Original:
builderr.err
sqluri = sqlite:////tmp/test.db
buildlog.log
Updated:
sqluri = mysql://user:password@server/database
where "user" is the current user on this install
"password" is the mysql password for this user
"server" is just localhost
"database" is a default or new database for use with the Sync server


Sync errors are generated by default due to missing user id info:
== Start the Sync Server ==
Could not get the user id for dd7j7agsa4acxnul6h6gpy5rfosncekz
* Run the following command (see the wiki page) for more information
This is probably OK.
$ bin/paster serve development.ini
or
$ nohup bin/paster serve development.ini 1>> syncserver.log 2>>syncserver.log &
While running, the Sync Server messages/events will print to STDOUT and STDERR.
This command will run the server outside of any login session and save STDOUT and STDERR to a log file.


Edits to ../etc/sync.conf
* If the Server is running correctly, you should see something like this
to point sqlite3 to temp.db in /home/mozroot rather than /tmp
Starting server in PID 3568.
This seems to work, except Sync errors are generated due to missing user id information.
serving on 0.0.0.0:5000 view at http://127.0.0.1:5000
This should be OK.


Now, trying to switch to MySQL:
= Running the Sync 1.1 Tests =
Fixed ../server-full/etc/sync.conf
* Once the build is complete, you can run the test as follows
sqluri = mysql://root:mozroot@localhost/syncdb
$ make test
* If everything runs as expected, you should see the following:
bin/nosetests -s --with-xunit deps/server-core/services/tests
deps/server-reg/syncreg/tests deps/server-storage/syncstorage/tests
............................................................etc.......
----------------------------------------------------------------------
Ran 177 tests in 33.031s
OK
* The tests are located in the following directories:
** server-full/deps/*/*/tests
* Note: Remember to cleanup after the test by deleted some db files in /tmp
rm /tmp/sync.db /tmp/tests*.db


Simply making this change causes the following errors:
= Restarting MySQL after System Restart =
$ bin/paster serve development.ini
* Possible lines for autostarting MySQL server on boot
Traceback (most recent call last):
$ sudo /etc/init.d/mysqld start ## use restart after update
  File "bin/paster", line 9, in <module>
## OR ##
    load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
$ sudo service mysqld start ## use restart after update
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run
$ sudo chkconfig --levels 235 mysqld on
    invoke(command, command_name, options, args[1:])
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run
    result = self.command()
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 276, in command
    relative_to=base, global_conf=vars)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 313, in loadapp
    **kw)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteDeploy-1.5.0-py2.7.egg/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteDeploy-1.5.0-py2.7.egg/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteDeploy-1.5.0-py2.7.egg/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteDeploy-1.5.0-py2.7.egg/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/PasteDeploy-1.5.0-py2.7.egg/paste/deploy/util.py", line 56, in fix_call
    val = callable(*args, **kw)
  File "/home/mozroot/server-full/deps/server-core/services/baseapp.py", line 320, in make_app
    app = klass(urls, controllers, params, auth_class)
  File "/home/mozroot/server-full/deps/server-storage/syncstorage/wsgiapp.py", line 101, in __init__
    auth_class)
  File "/home/mozroot/server-full/deps/server-core/services/baseapp.py", line 85, in __init__
    self.auth = None if auth_class is None else auth_class(self.config)
  File "/home/mozroot/server-full/deps/server-core/services/wsgiauth.py", line 53, in __init__
    self.backend = get_auth(self.config)
  File "/home/mozroot/server-full/deps/server-core/services/auth/__init__.py", line 233, in get_auth
    return ServicesAuth.get_from_config(config, 'auth')
  File "/home/mozroot/server-full/deps/server-core/services/pluginreg.py", line 138, in get_from_config
    return cls.get(backend_name, **config)
  File "/home/mozroot/server-full/deps/server-core/services/pluginreg.py", line 144, in get
    return klass(**params)
  File "/home/mozroot/server-full/deps/server-core/services/auth/sql.py", line 97, in __init__
    engine = create_engine(sqluri, **sqlkw)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/SQLAlchemy-0.6.8-py2.7.egg/sqlalchemy/engine/__init__.py", line 263, in create_engine
    return strategy.create(*args, **kwargs)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/SQLAlchemy-0.6.8-py2.7.egg/sqlalchemy/engine/strategies.py", line 66, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/home/mozroot/server-full/lib/python2.7/site-packages/SQLAlchemy-0.6.8-py2.7.egg/sqlalchemy/dialects/mysql/mysqldb.py", line 107, in dbapi
    return __import__('MySQLdb')
ImportError: No module named MySQLdb


As a final step, trying this again:
= Rebuilding and Deploying the Sync Server =
bin/easy_install MySQL-Python
* Kill the Sync Server process
$ kill -9 <PID>
(Leave MySQL running)
* Save the Sync Server directory and its contents
(this in case you want to keep various snapshots)
$ mv server-full server-full-<DATE>
(example: mv server-full server-full-08012011)
* Repeat the build and deploy steps from above
* Make the necessary changes in the server-full/etc/sync.conf file:
* Change the sqluri entries
* Restart the server using the "nohup" command from above
* Test the service to verify everything is running correctly


That seems to have gotten rid of the problem
= Redeploying a Live Server =
* The following steps can be used to redeploy a live sync server
$ cd /path/to/server-full
$ hg pull -uv
$ make build
$ apachectl graceful


Now, running FireFox:
* This should be sufficient, as "make build" will pull and update the repositories underneath the "deps" directory.
AND, we are good!
* There is also a "make update" target which will update only the dependent repositories, without rebuilding any of the third-party dependencies from PyPi.
* Also note that the Makefile will build the production channel by default, meaning this will not update you to the very latest checked-in changes but rather to the latest tags.  If instead you want to deploy the very latest code, then use this line instead:
$ make build CHANNEL=dev


* NOTE: More specific redeployment ideas can be found on this great site:
** http://www.wenks.ch/fabian/mozilla-custom-sync-server/#Updating%20the%20Sync%20Server




Also, the 7/11 build using MySQL does works!
For more advanced configurations, including web server support, please see the following documentation:
Saved out as server-full-BAK
* http://docs.services.mozilla.com/howtos/run-sync.html
For a more technical overview of Sync Server and an introduction to Mozilla Services, please see the following documentation:
* http://docs.services.mozilla.com




For more advanced configurations, including web server support, please see the following documentation: http://docs.services.mozilla.com/howtos/run-sync.html
Back To ==> [[QA/Browser_Technologies/Services | Mozilla BT Services]]
For a more technical overview of Sync Server and an introduction to Mozilla Services, please see the following documentation: http://docs.services.mozilla.com

Latest revision as of 22:08, 6 July 2012

Back To ==> Mozilla BT Services


Notes for Installing Sync Server on Fedora 15

REF: http://docs.services.mozilla.com/howtos/run-sync.html

General Notes

  • The "Building the server" steps can be repeated for each new build you want to test.
  • Once it is running, you can point to it for your own Sync server.
  • It is highly recommended that Python 2.6.6 be installed before proceeding with the Sync install.
  • Python 2.7 appears to be compatible, but the pre-packaged installer assumes 2.6.6.
  • Also, it is highly recommended that the developer tools are installed after installing Python 2.6.6.

Installing the Developer Tools

To get a full development environment on the box, do the following:

$ curl -O http://python-distribute.org/distribute_setup.py
$ sudo python2.6 distribute_setup.py
$ sudo easy_install-2.6 MoPyTools

Keeping the build logs

Run the "make build" step as follows:

$ make build 1> buildlog.log 2> builderr.err

Using Sqlite3

If sqlite3 is to be used, the server-full/etc/sync.conf file must be edited to change the default location of the temp.db file. The default location is in /tmp, which will get wiped at each system boot/restart.

Installing Sync Server

Assuming the prerequisites have been met, following the clone, build, and make steps from the link above.

Configuring Sync Server to use MySQL

The Sync Server can be re-configured to use MySQL instead of Sqlite3.

  • Install MySQL
$ sudo yum install mysql, mysql-devel, mysql-server

Note: mysql-python will also have to be installed since Fedora 15 does not pre-install this package.

$ cd server_full
$ bin/easy_install MySQL-Python
  • Start MySQL
$ sudo /etc/init.d/mysqld start
(will ask for the root password, if set)
will show something like the following:
Server version: 5.5.13 MySQL Community Server (GPL)
or
mysql  Ver 14.14 Distrib 5.5.14, for Linux (i686) using readline 5.1
  • Run the following to enter mysql:
$ mysql -u root -p
There may be no password yet (in which case the password field is blank)
  • If there is no password, run following to create one
mysql> UPDATE mysql.user set Password=PASSWORD('mozroot') WHERE User='root';
mysql> FLUSH PRIVILEGES
then
mysql> \quit
  • While in mysql, create a database for use with SyncServer
mysql> CREATE DATABASE syncdb;
mysql> \quit
  • Change two lines in the server-full/etc/sync.conf file
Original:
sqluri = sqlite:////tmp/test.db
Updated:
sqluri = mysql://user:password@server/database
where "user" is the current user on this install
"password" is the mysql password for this user
"server" is just localhost
"database" is a default or new database for use with the Sync server

Start the Sync Server

  • Run the following command (see the wiki page) for more information
$ bin/paster serve development.ini
or
$ nohup bin/paster serve development.ini 1>> syncserver.log 2>>syncserver.log &
While running, the Sync Server messages/events will print to STDOUT and STDERR.
This command will run the server outside of any login session and save STDOUT and STDERR to a log file.
  • If the Server is running correctly, you should see something like this
Starting server in PID 3568.
serving on 0.0.0.0:5000 view at http://127.0.0.1:5000

Running the Sync 1.1 Tests

  • Once the build is complete, you can run the test as follows
$ make test
  • If everything runs as expected, you should see the following:
bin/nosetests -s --with-xunit deps/server-core/services/tests
deps/server-reg/syncreg/tests deps/server-storage/syncstorage/tests
............................................................etc.......
----------------------------------------------------------------------
Ran 177 tests in 33.031s
OK
  • The tests are located in the following directories:
    • server-full/deps/*/*/tests
  • Note: Remember to cleanup after the test by deleted some db files in /tmp
rm /tmp/sync.db /tmp/tests*.db

Restarting MySQL after System Restart

  • Possible lines for autostarting MySQL server on boot
$ sudo /etc/init.d/mysqld start ## use restart after update
## OR ##
$ sudo service mysqld start ## use restart after update
$ sudo chkconfig --levels 235 mysqld on

Rebuilding and Deploying the Sync Server

  • Kill the Sync Server process
$ kill -9 <PID>

(Leave MySQL running)

  • Save the Sync Server directory and its contents

(this in case you want to keep various snapshots)

$ mv server-full server-full-<DATE>

(example: mv server-full server-full-08012011)

  • Repeat the build and deploy steps from above
  • Make the necessary changes in the server-full/etc/sync.conf file:
  • Change the sqluri entries
  • Restart the server using the "nohup" command from above
  • Test the service to verify everything is running correctly

Redeploying a Live Server

  • The following steps can be used to redeploy a live sync server
$ cd /path/to/server-full
$ hg pull -uv
$ make build
$ apachectl graceful
  • This should be sufficient, as "make build" will pull and update the repositories underneath the "deps" directory.
  • There is also a "make update" target which will update only the dependent repositories, without rebuilding any of the third-party dependencies from PyPi.
  • Also note that the Makefile will build the production channel by default, meaning this will not update you to the very latest checked-in changes but rather to the latest tags. If instead you want to deploy the very latest code, then use this line instead:
$ make build CHANNEL=dev


For more advanced configurations, including web server support, please see the following documentation:

For a more technical overview of Sync Server and an introduction to Mozilla Services, please see the following documentation:


Back To ==> Mozilla BT Services