Extending the Flask tutorial app into a fully-fledged, highly available, cloud-based application

If you’re starting out in Flask development, you’ll likely have followed the online tutorial over at pallet projects: https://flask.palletsprojects.com/en/1.1.x/tutorial/. This You may also be wondering how you’d go about deploying such an app for widespread use.

In this article, I’ll how you how to:

  • migrate the database from SQLite to PostgreSQL,
  • pretty up the user interface (this is totally optional, I just like to tweak the UI as we all do)
  • deploy the application on google cloud with Kubernetes, with scalability and database replication in mind.

Note: I’ve nothing against SQLite as a production database. A number of articles have appeared on the internet recently that positively puncture the myth that SQLite is a toy database. It’s not. However, for the purposes of this article, I’ll be using a PostgreSQL database to show how an application database is hosted separately from the application and accessed through a network interface.

Tools

Apart from an IDE and a command line to develop and test the application, you’ll need the google cloud tooling that will enable building and deploying the app to a Google Cloud Platform Kubernetes cluster:

  • The Google Cloud SDK (command-line interface),
  • kubectl installed as a gcloud component.

We’ll be using Kubernetes deployment descriptors directly for the purposes of this article. If you want to abstract away Google as a provider and allow your application to be readily portable to another cloud provider, you’d use Terraform or a similar tool that will abstract away the specifics of your cloud provider.

1. Shift from SQLite to PostgreSQL

SQLite and PostgreSQL are both relational databases, that is to say they store data in a table format and process reads, writes and updates through the SQL language. That said, SQL like any other standard suffers from vendor splintering and there are inevitably notable differences that will spring up in the SQL supported. Think of it as dialects.

To interface with the database from python within the Flask app, you will need a special connecter library. psycoph2 is used in this article to replace the SQLite database engine.

Here is the SQLite version of the schema to create the database:

DROP TABLE IF EXISTS user;
DROP TABLE IF EXISTS post;
CREATE TABLE user (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password TEXT NOT NULL
);
CREATE TABLE post (
id INTEGER PRIMARY KEY AUTOINCREMENT,
author_id INTEGER NOT NULL,
created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
body TEXT NOT NULL,
FOREIGN KEY (author_id) REFERENCES user (id)
);

And here is the PostgreSQL version:

DROP DATABASE IF EXISTS post;
CREATE DATABASE post;
\c post;
CREATE TABLE IF NOT EXISTS user (
id serial PRIMARY KEY,
username VARCHAR(255),
password VARCHAR(255)
);
CREATE TABLE IF NOT EXISTS post (
id serial PRIMARY KEY,
author_id INT,
created TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
title TEXT NOT NULL,
body TEXT NOT NULL,
FOREIGN KEY (author_id) REFERENCES user (id)
);

You’ll notice a few minute differences, notably the use of the keyword serial instead of autoincrement for incrementing the id primary key of each table.

The database interface can be hosted within a simple python file, which will then serve as a replacement for the existing db.py file.

The PostgreSQL version:

from os import environ
import psycopg2
import click
from flask import current_app, g
from flask.cli import with_appcontext
DATABASE_HOST = environ.get('DATABASE_HOST')
DATABASE_USERNAME = environ.get('DATABASE_USERNAME')
DATABASE_PASSWORD = environ.get('DATABASE_PASSWORD')
DATABASE_PORT = environ.get('DATABASE_PORT')
DATABASE_NAME = environ.get('DATABASE_NAME')
def get_db():
if 'db' not in g:
g.db = psycopg2.connect(
host=DATABASE_HOST,
user=DATABASE_USERNAME,
password=DATABASE_PASSWORD,
port=DATABASE_PORT,
dbname=DATABASE_NAME
)
return g.db
def close_db(e=None):
db = g.pop('db', None)
if db is not None:
db.close()
def init_db():
db = get_db()
with current_app.open_resource('schema.sql') as f:
db.execute(f.read().decode('utf8'))
@click.command('init-db')
@with_appcontext
def init_db_command():
"""Clear the existing data and create new tables."""
init_db()
click.echo('Initialized the database.')
@click.command('check-db')
@with_appcontext
def check_db_command():
"""Clear the existing data and create new tables."""
db = get_db()
cur = db.cursor()
cur.execute("select * from user")
users = cur.fetchall()
print(users)
cur.execute("select * from post")
posts = cur.fetchall()
print(posts)

def init_app(app):
app.teardown_appcontext(close_db)
app.cli.add_command(init_db_command)
app.cli.add_command(check_db_command)

You’ll notice that this database utility comes with some added environment variables. This is because the database will be accessed over the network.

You’ll notice I’ve chosen to get the database parameters directly from environment variables. This is different from the traditional method of having them in configuration files. Why? Because environment variables are easier to set and connect when working with containers. More on that later.

The click annotations are maintained in this version of the database connector because they will serve the same purpose: to harness application capabilities at runtime for database initialization and reset.

Moving from SQLite to PostgreSQL comes with a definite price in terms of development. The psycopg2 connector provides a different interface than the SQLite connector, so all methods that read/write/update/delete within the database have to be revisited. Notably, psycopg2 uses a cursor obtained from the db connection to execute SQL, but the dbconnection must also be visible within CRUD methods because it provides the commit() method that persists any database change.

Here is the SQLite version of the register method:

@bp.route('/register', methods=('GET', 'POST'))
def register():
if request.method == 'POST':
username = request.form['username']
password = request.form['password']
db = get_db()
error = None
if not username:
error = 'Username is required.'
elif not password:
error = 'Password is required.'
elif db.execute(
'SELECT id FROM user WHERE username = ?', (username,)
).fetchone() is not None:
error = 'User {} is already registered.'.format(username)
if error is None:
db.execute(
'INSERT INTO user (username, password) VALUES (?, ?)',
(username, generate_password_hash(password))
)
db.commit()
return redirect(url_for('auth.login'))
flash(error)return render_template('auth/register.html')

And there is the PostgreSQL version. Notice how separate objects are used for querying and commiting:

@bp.route('/register', methods=('GET', 'POST'))
def register():
if request.method == 'POST':
username = request.form['username']
password = request.form['password']
db = get_db()
cur = db.cursor()
error = None
if not username:
error = 'Username is required.'
elif not password:
error = 'Password is required.'
else:
db = get_db()
cur = db.cursor()
cur.execute(
'SELECT id FROM user WHERE username = %s', (username,))
if cur.fetchone() is not None:
error = 'User {} is already registered.'.format(username)
if error is None:
cur.execute(
'INSERT INTO user (username, password) VALUES (%s, %s)',
(username, generate_password_hash(password))
)
db.commit()
return redirect(url_for('auth.login'))
flash(error)return render_template('auth/register.html')

It should be noted that coding SQL statements directly into python code is not something most developers will do. Libraries such as SQLAlchemy allow the dev to abstract away the database through ORM, or Object Relational Mapping. I won’t go into the details of such an implementation, given that this article means to show how a migration from one database to another can be done with minimal impact to the code.

It’s not technically necessary to change the UI of the app to show how to scale it into a high availability cloud infrastructure, but I’ve included it anyway, if only to highlight what I consider an important point in application development. To your end user, the application is very much defined by the way it looks and feels. Whatever the backend may be, the front-facing UI is an app’s true identity.

How much you change or not is up to you. To do so, simply modify the flaskr/static/style.css file:

html {
font-family: monospace;
background: #222;
color: #ccc;
padding: 1rem;
}
body {
max-width: 960px;
margin: 0 auto;
border-radius: 1.4em;
}
h1 {
font-family: monospace;
color: #777;
font-size: 300%;
margin: 1rem 0;
}
.button {
border: none;
padding: 0.8em;
}
input {
padding: 0.8em;
}
a, .button {
color: #FF0;
background-color: #760;
border-radius: 0.4em;
text-decoration: none;
font-size: 120%;
margin: 0.2em;
}
/* you get the point */

Running the application on a Kubernetes cluster means running it within containers. To do this, you will need a Docker file and docker execution environment. Google Cloud Builds provides means of creating Docker images on its own infrastructure.

Here is the docker file to install the application on a python image:

FROM python:3.6-jessie
RUN apt update
WORKDIR /flask-blog
ADD app/requirements.txt /flask-blog/requirements.txt
RUN pip install -r /flask-blog/requirements.txt
ADD app /flask-blog
CMD ["flask", "run", "--host", "0.0.0.0"]
EXPOSE 5000

Note that I’m still using the development flask run method. You would want to switch to something more sturdy for production purposes. For example, the Flask tutorial recommends using the waitress tool.

Using the Google Cloud Platform command-line interface gcloud, leverage the builds component to build the image remotely on GCP and tag it for later use:

gcloud builds submit - tag gcr.io/<your project name>/flask-blog:v1

In the Google Cloud Console, navigate to flask-blog within the container registry and copy the full name of the image created. When deploying the app with Kubernetes, the full name will be used with the kubectl deployment descriptor to create containers from the image.

The database will run inside a separate container, so it will need its own docker image. The Dockerfile shown here uses a “latest” image from Postgres, which is not recommended as you don’t know which version you’re using. It’s always better to use a precise version of the database to ensure you know exactly what you’re using, how known issues might affect you and what kind of technical debt you’ve incurred.

Database Dockerfile:

FROM postgres:latest
COPY schema.sql /docker-entrypoint-initdb.d/schema.sql
EXPOSE 5432
ENV POSTGRES_PASSWORD <your password>

To build this image on GCP:

gcloud builds submit - tag gcr.io/<your project name>/flask-blog-db:v1

You’ll be deploying your application into a Kubernetes cluster. Your first step is therefore to create the cluster environment. This can be done with a single command:

gcloud container clusters create blog-cluster

Note that doing things this way preempts any specific configuration of the cluster that you may want to leverage to lower the cost of your infrastructure. GCP proposes a low-cost “my-first-cluster” configuration that is much cheaper to use for development or exploration purposes.

You must now connect your Kubernetes configuration to the remote cluster. To do so, you will fetch the cluster credentials into your local Kubernetes environment:

gcloud container clusters get-credentials blog-cluster

So all that’s left is to deploy the application. Simple, eh? Well, you have to deploy more than just the containers and hope they can see each one another. Using services, you can define network specific network locations within the cluster.

The deployment descriptor we’ll be using specifies three containers hosted under a load balancer for high availability. This ensures that the application will readily scale to a great number of users.

Here are the elements of the deployment descriptor. These elements can all be grouped within the same file or kept separate.

The database service descriptor:

## the database service allowing for connecting to the 
## main database by name and not IP
apiVersion: v1
kind: Service
metadata:
name: db-service
spec:
selector:
app: flask-blog-db-container
ports:
- protocol: TCP
port: 5432
targetPort: 5432

This service means that all containers tagged with flask-blog-db-container will be accessible through the db-service name.

The port remains unchanged, mapping port 5432 to 5432.

The database descriptor:

## The database container
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: flask-blog-db
spec:
serviceName: flask-blog-db-container
replicas: 1
selector:
matchLabels:
app: flask-blog-db-container
template:
metadata:
labels:
app: flask-blog-db-container
spec:
containers:
- name: db
image: gcr.io/<your project name>/<your database image full name>
ports:
- containerPort: 5432
name: postgredb

Notice the database is using a StatefulSet. You’d want to connect this to block storage with the proper directory structure linked so that the persistence of the database is ensured if anything goes wrong with it: this allows a new database to respawn and take up where it left off.

Next, we create a load balancer for the front end of the application:

## the externally-exposed load balancer 
## enabling connection to the application
apiVersion: v1
kind: Service
metadata:
name: flask-blog-front
spec:
selector:
app: flask-blog
ports:
- protocol: TCP
port: 80
targetPort: 5000
type: LoadBalancer

The actual containers will be listening on port 5000 but the load balancer will be listening on the more usual port 80. This port mapping shows how the container configuration can be quite neatly decoupled from the runtime infrastructure with simple mapping mechanisms such as this one.

Finally, the descriptor for the application, which uses a 3-way replica:

## the application deployment 
## for clustering three replicas
## of the application
apiVersion: apps/v1
kind: Deployment
metadata:
name: flask-blog-deployment
labels:
app: flask-blog
spec:
replicas: 3
selector:
matchLabels:
app: flask-blog
template:
metadata:
labels:
app: flask-blog
spec:
containers:
- name: blog-container
image: gcr.io/<your project name>/<app container image full name>
env:
- name: FLASK_APP
value: "flaskr"
- name: FLASK_ENV
value: "development"
- name: DATABASE_HOST
value: "db-service"
- name: DATABASE_USERNAME
value: "postgres"
- name: DATABASE_PASSWORD
value: "<your password>"
- name: DATABASE_PORT
value: "5432"
- name: DATABASE_NAME
value: "blogs"

Notice how the environment variables used to configure the database access within the app are set directly into the container from the Kubernetes deployment descriptor. The address of the database host is not known at runtime because the ip will be set once the container is running, but the database will be network accessible within the cluster by it’s service name.

The last step is to run the deployment by applying all the files containing Kubernetes configuration:

kubectl apply -f <deployment descriptor file>.yaml

Execute this last step for every deployment descriptor.

Cloud deployment has its pros and cons, of course. But I must admit, as someone who’s been in the IT field for a good many years, that simply running a few commands from the comfort of my home and having a whole application up and running and accessible on the internet, retains a certain wondrous quality.

I hope you’ve enjoyed this article, and I should apologize that in the interest of showing an entire migration chain in a single article, a good many elements were occluded and that piecing together the various steps will require some work if you’re not already familiar with the tools used.

I write code, stories, and songs. https://www.reneghosh.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store