Deploying PYBOSSA with nginx and uwsgi

PYBOSSA is a python web application built using the Flask micro-framework.

This guide describes how to make PYBOSSA run as a service or daemon permanently in the background. This is useful if you want to run a production ready single PYBOSSA web server. If you only want to test PYBOSSA please follow only install.

Pre-requisites:

  • nginx
  • uwsgi
  • supervisord
  • Redis and sentinel as service (with supervisord)
  • RQ-Scheduler and RQ-Worker as service (with supervisord)
  • PYBOSSA as service (with supervisord)

First steps

If you have not done already: Please create a new user account, e.g. pybossa (a non root at best) which will run the PYBOSSA instance. You then have to follow the instructions from install first to get a runnable PYBOSSA. After you’ve done that please continue with this guide.

Installing nginx and uwsgi

You have to install nginx and uwsgi in your server machine. In a Debian/Ubuntu machine you can install them running the following commands:

1
sudo apt-get install nginx

in the (virtualenv-)installation directory of pybossa you need to install uwsgi:

1
pip install -U uwsgi

Configuring nginx and uwsgi for PYBOSSA

We assume you only want to run PYBOSSA on your nginx webserver. If you want to run also other services on the same server you need to modify the nginx config files!

You have to copy and adapt the following files from your local PYBOSSA installation:

  • contrib/nginx/pybossa
  • contrib/pybossa.ini

The PYBOSSA virtual host file (contrib/nginx/pybossa) has the following directives:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
location / { try_files $uri @pybossa; }

location @pybossa {
    include uwsgi_params;
    uwsgi_pass unix:/tmp/pybossa.sock;
}

location  /static {

            # change that to your pybossa static directory
            alias /home/Scifabric/pybossa/pybossa/themes/default/static;

            autoindex on;
            expires max;
        }

You can specify a user and group from your machine with lower privileges in order to improve the security of the site. You can also use the www-data user and group name.

Once you have adapted the PATH in the alias in that file, copy it into the folder:

1
sudo cp contrib/nginx/pybossa /etc/nginx/sites-available/.

Please delete the default config in sites-enabled (do not worry there is a backup):

1
sudo rm /etc/nginx/sites-enabled/default

Enable the PYBOSSA site:

1
sudo ln -s /etc/nginx/sites-available/pybossa /etc/nginx/sites-enabled/pybossa

And restart the server:

1
sudo service nginx restart

Creating the pybossa.ini file for uwsgi

You have to copy the pybossa.ini.template file to pybossa.ini in your PYBOSSA installation and adapt the paths to match your configuration!

The content of this file is the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
[uwsgi]
socket = /tmp/pybossa.sock
chmod-socket = 666
chdir = /home/pybossa/pybossa
pythonpath = ..
virtualenv = /home/pybossa/pybossa/env
module = run:app
cpu-affinity = 1
processes = 2
threads = 2
stats = /tmp/pybossa-stats.sock
buffer-size = 65535

Install supervisord

Supervisord is used to let PYBOSSA and its RQ system run as Daemon in the background. It shares some of the same goals of programs like launchd, daemontools, and runit.

Install it:

1
sudo apt-get install supervisor

Configure Redis and sentinel as service with supervisord

First stop redis service and all running redis instances with:

1
2
sudo service redis-server stop
killall redis-server

We want to run redis and sentinel with supervisord because supervisord is more reliable when redis crashes (which can happen when you have too less memory). So we disable redis-server daemon service with:

1
sudo rm /etc/init.d/redis-server

Go to your pybossa installation directory and copy following files:

1
2
3
4
5
6
sudo cp contrib/supervisor/redis-server.conf /etc/supervisor/conf.d/
sudo cp contrib/supervisor/redis-sentinel.conf /etc/supervisor/conf.d/
sudo cp contrib/redis-supervisor/redis.conf /etc/redis/
sudo cp contrib/redis-supervisor/sentinel.conf /etc/redis/
sudo chown redis:redis /etc/redis/redis.conf
sudo chown redis:redis /etc/redis/sentinel.conf

Now we restart supervisord (please do a full stop and start as described):

1
2
sudo service supervisor stop
sudo service supervisor start

To verify install you can list all redis processes and you should see a redis-server at port 6379 and redis-sentinel at port 26379:

1
ps aux | grep redis

This two services will no run whenever the server is running (even after reboot).

Configure RQ-Scheduler and -Worker to run with supervisord

You need to adjust the paths and user account in this two config files according to your installation! Then copy them to supervisor (do not forget to edit them):

1
2
sudo cp contrib/supervisor/rq-scheduler.conf.template /etc/supervisor/conf.d/rq-scheduler.conf
sudo cp contrib/supervisor/rq-worker.conf.template /etc/supervisor/conf.d/rq-worker.conf

Restart supervisor fully:

1
2
sudo service supervisor stop
sudo service supervisor start

Verify service running. You should see a rqworker and rqscheduler instance in console:

1
ps aux | grep rq

Setup PYBOSSA itself

This steps are recommended to do when you run PYBOSSA in nginx. Open your settings_local.py in your PYBOSSA installation and uncomment or delete the two lines with HOST and PORT, e.g.:

1
2
# HOST = '0.0.0.0'
# PORT = 12000

After that specify the full server URL where your PYBOSSA is reachable, e.g.:

1
2
SERVER_NAME = mypybossa.com
PORT = 80

Let PYBOSSA run as service

Finally we need to let pybossa run as service. Adjust again the paths and user name in this file and copy it to supervisor config directory:

1
sudo cp contrib/supervisor/pybossa.conf.template /etc/supervisor/conf.d/pybossa.conf

Edit now the file and adjust paths & user name.

Restart supervisor fully:

1
2
sudo service supervisor stop
sudo service supervisor start

You should now have a running PYBOSSA production ready webserver on your nginx. Open your browser and check your configured domain http://example.com.

Congratulations! 😄

How to update PYBOSSA service

Upgrading and updating PYBOSSA as service works the same as for a standalone version. Please follow instructions on install. However a few extra steps are required after you updated.

You need to restart all supervisor controlled services after updating:

1
2
3
sudo supervisorctl restart rq-scheduler
sudo supervisorctl restart rq-worker
sudo supervisorctl restart pybossa

Logs of PYBOSSA services

You can find logs of all PYBOSSA services in this directory:

1
cd /var/log/supervisor

Last words about Security and Scaling

This guide does not cover how to secure your PYBOSSA installation. As every web server you have to make it secure (like e.g. strong passwords, automatic Ubuntu security updates, firewall, access restrictions). Please use guides on the Internet to do so.

PYBOSSA can also be scaled horizontally to run with redundant servers and with zero downtime over many redis, db and web servers with load balancers in between.

If you need a secure and/or scalable PYBOSSA installation please contact us. We will be glad to help you and we can even do all the hosting, customization, administration and installation for you. Check our pricing page and get in touch.