#devops

How to Host a Python App on AWS EC2

Table of Contents

• Access the server:
• Install git:
• Install pip:
• Create project folder:
• Create virtualenv:
• Install nginx
  • (Linux AMI)
  • (Amazon Linux 2023)
• Start nginx
• Create file supervisor conf:
• Install supervisor
  • (Linux AMI)
  • (Amazon Linux 2023)
• test (up the server standalone)
• Startup Nginx
• Script supervisord
• Startup script
• Install certbot
  • (Linux AMI)
  • (Amazon Linux 2023)
• Generate the certificate to the current domain in nginx
• Install cron for Amazon 2023
• Renew automatically with crontab
• Visualize the active crons on the machine

Require a new review!

Access the server:

chmod 400 path/to/AccessKey.pem
  ssh -i path/to/AccessKey.pem ec2-user@****.compute-1.amazonaws.com

Install git:

yum install git

Install pip:

curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py --user

Create project folder:

mkdir /var/www
git clone <myrepo>
cd <myrepo>

Create virtualenv:

  python3 -m venv .
source bin/activate

Install nginx

(Linux AMI)

sudo amazon-linux-extras install nginx1

(Amazon Linux 2023)

sudo dnf update
sudo dnf install -y nginx

Start nginx

systemctl start nginx.service

/etc/nginx/nginx.conf

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;

}

/etc/nginx/sites-enabled/mysite.conf

NOTA: Disable the SSL for using the new certbot to generate a certificate and to modify the file to us.

upstream app_server {
        server unix://www/mysite/run/gunicorn.sock fail_timeout=0;
}

server {
    listen         80;
    server_name    mysite.com.br;
    return         307 https://$host$request_uri;
}

server {
    listen       443 ssl http2;
    listen       [::]:443 ssl http2;
    server_name  mysite.com.br;

    ssl_certificate "/etc/letsencrypt/live/mysite.com.br/fullchain.pem";
    ssl_certificate_key "/etc/letsencrypt/live/mysite.com.br/privkey.pem";
    ssl_trusted_certificate "/etc/letsencrypt/live/mysite.com.br/fullchain.pem";
    ssl_dhparam "/etc/letsencrypt/live/mysite.com.br/dhparams.pem";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    # ssl_ciphers PROFILE=SYSTEM;
    # ssl_prefer_server_ciphers on;


    client_max_body_size 4G;

    access_log /www/mysite/logs/nginx-access.log;
    error_log /www/mysite/logs/nginx-error.log;

    location /static/ {
        alias   /www/mysite/static_cdn/;
    }

    location /media/ {
        alias   /www/mysite/media/;
    }

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        if (!-f $request_filename) {
                proxy_pass http://app_server;
                break;
        }
    }

}

Create file supervisor conf:

/etc/supervisord.conf

[inet_http_server]
port=*:9001
username=root
password=LFn6d92TBS3bemHD

[supervisord]
logfile=/tmp/supervisord.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB        ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10           ; # of main logfile backups; 0 means none, default 10
loglevel=info                ; log level; default info; others: debug,warn,trace
pidfile=/tmp/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false               ; start in foreground if true; default false
minfds=1024                  ; min. avail startup file descriptors; default 1024
minprocs=200                 ; min. avail process descriptors;default 200
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
[program:mysite]
command = /www/mysite/gunicorn_start.sh
user = root
stdout_logfile = /www/mysite/logs/gunicorn_supervisor.log
redirect_stderr = true
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
autorestart = false

Install supervisor

(Linux AMI)

sudo amazon-linux-extras install epel
yum install supervisor

(Amazon Linux 2023)

# required pip and virtualenv activated
pip install supervisor

test (up the server standalone)

bin/supervisord -n 

Startup Nginx

systemctl enable nginx
chkconfig nginx on

Script supervisord

/etc/init.d/supervisord

#!/bin/bash
#
# supervisord   Startup script for the Supervisor process control system
#
# Author:       Mike McGrath <mmcgrath@redhat.com> (based off yumupdatesd)
#               Jason Koppe <jkoppe@indeed.com> adjusted to read sysconfig,
#                   use supervisord tools to start/stop, conditionally wait
#                   for child processes to shutdown, and startup later
#               Erwan Queffelec <erwan.queffelec@gmail.com>
#                   make script LSB-compliant
#
# chkconfig:    345 83 04
# description: Supervisor is a client/server system that allows \
#   its users to monitor and control a number of processes on \
#   UNIX-like operating systems.
# processname: supervisord
# config: /etc/supervisord.conf
# config: /etc/sysconfig/supervisord
# pidfile: /var/run/supervisord.pid
#
### BEGIN INIT INFO
# Provides: supervisord
# Required-Start: $all
# Required-Stop: $all
# Short-Description: start and stop Supervisor process control system
# Description: Supervisor is a client/server system that allows
#   its users to monitor and control a number of processes on
#   UNIX-like operating systems.
### END INIT INFO

# Source function library
. /etc/rc.d/init.d/functions

# Source system settings
if [ -f /etc/sysconfig/supervisord ]; then
    . /etc/sysconfig/supervisord
fi

# Path to the supervisorctl script, server binary,
# and short-form for messages.
supervisorctl=/usr/local/bin/supervisorctl
supervisord=${SUPERVISORD-/www/mysite/bin/supervisord}
prog=supervisord
pidfile=${PIDFILE-/tmp/supervisord.pid}
lockfile=${LOCKFILE-/var/lock/subsys/supervisord}
STOP_TIMEOUT=${STOP_TIMEOUT-60}
OPTIONS="${OPTIONS--c /etc/supervisord.conf}"
RETVAL=0

start() {
    echo -n $"Starting $prog: "
    daemon --pidfile=${pidfile} $supervisord $OPTIONS
    RETVAL=$?
    echo
    if [ $RETVAL -eq 0 ]; then
        touch ${lockfile}
        $supervisorctl $OPTIONS status
    fi
    return $RETVAL
}

stop() {
    echo -n $"Stopping $prog: "
    killproc -p ${pidfile} -d ${STOP_TIMEOUT} $supervisord
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -rf ${lockfile} ${pidfile}
}

reload() {
    echo -n $"Reloading $prog: "
    LSB=1 killproc -p $pidfile $supervisord -HUP
    RETVAL=$?
    echo
    if [ $RETVAL -eq 7 ]; then
        failure $"$prog reload"
    else
        $supervisorctl $OPTIONS status
    fi
}

restart() {
    stop
    start
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status -p ${pidfile} $supervisord
        RETVAL=$?
        [ $RETVAL -eq 0 ] && $supervisorctl $OPTIONS status
        ;;
    restart)
        restart
        ;;
    condrestart|try-restart)
        if status -p ${pidfile} $supervisord >&/dev/null; then
          stop
          start
        fi
        ;;
    force-reload|reload)
        reload
        ;;
    *)
        echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload}"
        RETVAL=2
    esac

    exit $RETVAL

Startup script

chmod a+x /etc/init.d/supervisord
ps -fe | grep supervisor
sudo chkconfig --add supervisord
sudo chkconfig supervisord on

Install certbot

(Linux AMI)

sudo amazon-linux-extras install epel
yum install certbot-apache

(Amazon Linux 2023)

sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip
sudo /opt/certbot/bin/pip install certbot certbot-nginx
sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot

Ensure that the domain does not refer to another server with https. Because the certbot uses the nginx to found out the domain and change the mysite.conf. In the other words, DO NOT insert the SSL certificate definitions manually in the file. Leave certbot do it.

Generate the certificate to the current domain in nginx

sudo certbot --nginx

Install cron for Amazon 2023

sudo yum install cronie -y
sudo systemctl enable crond.service
sudo systemctl start crond.service
sudo systemctl status crond.service

Renew automatically with crontab

crontab -e
0 0 * * * root sudo certbot renew -q

Visualize the active crons on the machine

crontab -l

Domain for supervisor (in general, the subdmain of amazon).

/etc/nginx/sites-enabled/supervisord.conf

upstream supervisord {
        server localhost:9001 fail_timeout=0;
}

server {
        listen 80;
        server_name <host>.compute-1.amazonaws.com;
        access_log /var/log/access_supervisor.log;
        error_log /var/log/error_supervisor.log;

        location / {

                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_redirect off;

                proxy_pass http://supervisord;
        }

}