Build your Python project right way — Python, Docker, CentOS

Believe in simplicity and keep coding calm.

Image for post
Image for post
Photo by David van Dijk on Unsplash

We are going to see how to build a Python project and Dockerize it with bare minimum steps. The end goal was to get a Docker Image which can be run on-demand with parameters.

Case Study

For this tutorial, we are going to develop a very small application in Python 3. The application will take two numbers and one algebraic operand and will return the output of that operation.

<application> 10 + 20
# it shall return 30
<application> 10 * 20
# it shall return 200

Quite simple and yet intuitive. Let’s dive in and we will build a deployable/runnable Docker Image with the python project.

First, we need a plan

Image for post
Image for post
Photo by Estée Janssens on Unsplash

1 hour of good planning can save 10 hours of effort.

We first need to establish our new architecture for this project. We are going to develop a Python 3 distribution project which will later build by Dockerfile. The following is what we will start with.

myApp
├── .gitignore
├── Dockerfile
├── MANIFEST.in
├── app
│ ├── bin
│ └── lib
├── setup.py
└── startup.sh

Few unknown things in the above structure.

  • MANIFEST.in : This file is like a MANIFEST in JAR. It stores the information about flat files (non-python files) and informs the building process to include them in packaging.
  • startup.sh : This file will be our entry point when the project will get called from the outer world inside a Docker container. We can ignore it while in the development stage.

For lazy a**es like me, I am putting the code alongside for you guys to copy.

mkdir myapp
cd myapp
touch MANIFEST.in setup.py Dockerfile .gitignore startup.sh
mkdir -p app app/bin app/lib

All our code pointing to the outer world will be placed under the app/bin directory, where all other code we will use app/lib directory for.

The __init__ thing!

Python got its style while defining modules. If you have a bunch of scripts doing similar things inside a directory, just place an empty __init__.py file inside that directory and it now turned to a Python recognized module. In our example, we want app and app/bin, app/lib to become parent and child modules.

touch app/__init__.py app/bin/__init__.py app/lib/__init__.py

And we are done here!

Print the first python line

Image for post
Image for post
Photo by Max Duzij on Unsplash

The first line of the project always a big deal. We know it, you know it. We will start small and later jump big. Let us create a file, put some dummy lines to print and see how it goes.

touch app/bin/myapp.py

Let’s write some code inside myapp.py

# myapp.pydef main():    print('this is my first line of code')if __name__ == "__main__":    main()

But we cannot run it now. Before that, we need to do something with our environment. Let’s go back and complete our preparation stage.

Virtual Environments

Image for post
Image for post
Photo by Oleksii Hlembotskyi on Unsplash

Before starting anything beyond, we need our environment to be project-specific and we will not want to clutter our machine’s environment variables and binaries. Enter, virtualenv.

pip3 install virtualenv
cd myapp
virtualenv -p python3 env

Here we created a Python environment env. If you check your project folder, you will find a new directory there with the same name. Now this directory becomes our Python in that system. If this environment is activated, code will not go further and your system’s environment stays clean and unharmed. To activate it,

source env/bin/activate

(Resume) Print the first python line

Now we are set up with our environment. Let us now run our code.

python app/bin/myapp.pythis is my first line of code

Cool, but we have just started. We need our project to start accepting parameters. We will use a Python package argparse to deal with our parameter requirement.

Argument handling — welcome ‘argparse’

pip install argparse

Let’s go back to our code and write a few lines to parse the arguments. Here we need three arguments to our code.

  • Number 1
  • Operand
  • Number 2

So, we write something like this:

import argparsedef main():
print(‘this is my first line of code’)
parser = argparse.ArgumentParser(description=’Calculator’)
parser.add_argument(‘number1’, metavar=’NUMBER’, type=int, help=’First NUMBER’)
parser.add_argument(‘operand’, metavar=’OPERAND’, type=str, help=’OPERATION TO PERFORM’)
parser.add_argument(‘number2’, metavar=’NUMBER’, type=int, help=’Second NUMBER’)
args = parser.parse_args()
print(args.number1, args.operand, args.number2)
if __name__ == “__main__”:
main()

We also added an extra print line to see what we are receiving as a parameter.

python app/bin/myapp.py 10 + 20this is my first line of code
10 + 20

Well and good. We need the operand thing to make work. We will create a new class as Operation and add methods for various algebraic operations. We will place it under app/lib.

object-oriented programming

Image for post
Image for post
Photo by Malcolm Lightbody on Unsplash
touch app/lib/operation.py

What inside the file!

# operation.pyclass Operation:def __init__(self):
super().__init__()
def sum(self, number1, number2):
return number1 + number2
def sub(self, number1, number2):
return number1 - number2
def multiply(self, number1, number2):
return number1 * number2
def divide(self, number1, number2):
return number1 / number2 if not (number2 == 0) else 0

Now we have our helper class Operation functional. Now we are going to use it in our code in the app/bin directory.

from app.lib.operation import Operation

As Python does not support case, let us write an if..else to match operand parameter and methods under Operation class. Let’s add a few lines in myapp.py file.

def main():  operation = Operation()
result = 0
if args.operand == '+':
result = operation.sum(args.number1, args.number2)
elif args.operand == '-':
result = operation.sub(args.number1, args.number2)
elif args.operand == '/':
result = operation.divide(args.number1, args.number2)
elif args.operand == '*':
result = operation.multiply(args.number1, args.number2)
print(args.number1, args.operand, args.number2, '=', result)

Save it and let’s run the file again in the same way as last time.

python app/bin/myapp.py 10 + 20Traceback (most recent call last):
File "app/bin/myapp.py", line 2, in <module>
from app.lib.operation import Operation
ModuleNotFoundError: No module named 'app'

I hope we both stumbled to the same error! It is saying python does not know the module app that we have used in our import statement. What is the problem here?

We have defined virtualenv, activated it but we have not installed our python project inside that environment yet. Hence Python in that environment does not have any knowledge of the app module.

python setup.py install

Nothing happened! Because we have nothing inside setup.py.

setuptools

Python packaging world split into two binary distributions — Egg and wheel. setuptools is a python package that helps you to build your python project and get binaries in the desired format. We are not looking for binaries here. All we want is to build our project and run it under our virtual environment. Let’s write the code.

# setup.pyfrom setuptools import find_packages, setupsetup(
name='myapp',
version='0.0.1',
description='myapp description',
author='Arghajit',
author_email='dummy@dummy.com',
python_requires='>=3.6.0',
url='https://github.com/me/myproject',
packages=find_packages(),
install_requires=[
'argparse'
])

Now we run it again.

python setup.py install

This time something happened. The above command actually does 3 things for you.

  • Fetch requirements for your project that you have specified in install_requires at setup.py and install them inside the virtual environment.
  • Creates the distribution (package).
  • Installs the distribution.

Now let us run the code again and see.

python app/bin/myapp.py 10 + 20this is my first line of code
10 + 20 = 30

Let us remove that line now and try one more time.

python app/bin/myapp.py 10 + 2010 + 20 = 30

Done!

Dockerize the project

Image for post
Image for post
Photo by Guillaume Bolduc on Unsplash

Choosing the right Docker image can always be a challenge. I myself prefer using centos docker image as the base image and later modify the image as per own need — a safe bet!

# DockerfileFROM centos:8

Install binaries

RUN yum install -y epel-release sudo jq gcc python3 python3-devel

Create a directory for the project code, set it as the working directory.

RUN mkdir /etc/app
WORKDIR /etc/app

Copy the project code

COPY . /etc/app

Install Pip

Pip installation often becomes daunting. We go with the simplest and cleanest approach.

RUN curl “https://bootstrap.pypa.io/get-pip.py" -o “get-pip.py”
RUN python3 get-pip.py

Install virtualenv and prepare the environment and activate it.

RUN pip install virtualenv
RUN virtualenv -p python3 env
RUN source env/bin/activate

Build and install the project.

RUN python3 setup.py install

Entry Point

Finally, the entry point! To run a script with a passing parameter while calling a pre-build docker container, we need a script (ideally bash or anything) that will translate the parameters and start execution of your desired command. Here startup.sh is the file we are using. Let us write the file.

#!/usr/bin/env bashexec python3 app/bin/myapp.py $@

Here the $@ signifies all parameters which are passed when this bash file was called. We are passing them down to our python code.

We changed python to python3 to be on the safer side we are calling the correct python version.

All we want is, whenever the soon-to-be-build docker image going to be called, internally our above python code shall get executed. We write our final line at Dockerfile and saves it.

ENTRYPOINT [“/bin/bash”,”./startup.sh”]

Build the Docker Image

To run build an image out of Dockerfile, we go to the residing directory and run the following.

docker build -t myapp .

Run the Docker Image

Once you get a successful build, you can check your list of available docker images with the following command.

docker images

You should myapp listed there. Once you confirm the same, run the image with parameter as the python script.

docker run -t myapp 10 + 2010 + 20 = 30

And we are done!

We got a ready docker image with our python code inside which can be run anytime, anywhere.

I am not going to share the full code. Do it yourself, comrades.

Technical Member in Product Company, Agile practitioner.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store