Use Docker to package your Conda environment and Flask app. Dockerize your Python Flask app and recreate your Conda Python environment.
Docker, Conda, and Flask work together
This post will show how to use Docker together with Conda and your Python Flask app to recreate your Conda Python environment and run the app.
So you are developing a Python Flask app, and you have set a Conda virtual environment up on your local machine to run your app. Now you want to put your Python Flask app in a Docker image.
Wouldn’t it be nice if you could export your current Conda environment as a .yml file, describing what Python version your application is using and which Python libraries are required to run your application? Furthermore, use the exported .yml file to build a similar environment in a Docker image and run your Flask app in that environment.
The following will describe exactly how you can accomplish all the above mentioned.
If you specifically want to know more about how to use Conda and python for setting up an environment for working on data science tasks, then head over to my post: Python Data Science Environment.
Requirements
- You have Docker installed and running
- Some familiarity with Docker commands. You can read more about Docker here.
- Some familiarity with Conda commands. You can read more about Conda here.
Example Flask App
Throughout this post I will use the following mini Flask application.
# -*- coding: utf-8 -*- # # Imports # from flask import Flask, jsonify app = Flask(__name__) # # Routes # @app.route('/', methods=['GET']) def hello_world(): return jsonify({'message': 'Hello World'}) @app.route('/test', methods=['GET']) def test(): return jsonify({'test': 'test'}) if __name__ == "__main__": app.run(debug=True) # remember to set debug to False
Create a folder where you want your project files to be. I have created a folder called ‘docker_conda_template’ on my local machine. In that folder, I have created a file called ‘api.py’ which contains the above Python code.
Exporting the Conda Python environment
To recreate a Conda python environment in the docker image, which corresponds to the Conda python environment we have created on our local machine, we first need to export an environment.yml file from our local Conda Python environment.
To do that you first need to cd into the folder where you want your environment.yml file to be created.
cd home/docker_conda_template
Now we want to activate a Conda environment that has the necessary Python version installed and has all the libraries installed required to run our app. I assume you are familiar with the Conda CLI, therefore this post will not explain how to use Conda.
conda source activate your-environment-name
To export your environment into a .yml file run the following command.
conda env export > environment.yml
This will create the file ‘environment.yml’ in your current folder.
The file will look something like this.
name: your-environment-name
channels:
- defaults
dependencies:
- python=3.6
- flask
- gunicorn
This file contains three pieces of information. The name of the environment when it is created, what channels the libraries will be downloaded from, and finally what Python version and libraries to install.
Dockerfile for building your Docker image
The Dockerfile is where we put it all together and this is also the file that Docker will use to create an image with your Python application and all the requirements it needs to run e.g. our Conda environment as defined in our environment.yml file.
The following is the final Dockerfile:
FROM continuumio/miniconda:latest
WORKDIR /home/docker_conda_template
COPY environment.yml ./
COPY api.py ./
COPY boot.sh ./
RUN chmod +x boot.sh
RUN Conda env create -f environment.yml
RUN echo "source activate your-environment-name" > ~/.bashrc
ENV PATH /opt/conda/envs/your-environment-name/bin:$PATH
EXPOSE 5000
ENTRYPOINT ["./boot.sh"]
We are not going to define how to install Conda on the image, for that purpose we will download and use an existing image that already has Conda installed and set up. This image is provided by continuum, it has Miniconda installed and set up. We will use their image as a basis for building ours. In order to use their image as a base image, we must state that in the Dockerfile. This is done in the first line “FROM continuumio/miniconda:latest”.
To define a working directory in your image, we state “WORKDIR /home/docker_conda_template”.
We also want to copy files such as our code and environment.yml file into the image. The following lines copy the file into the Docker image.
COPY environment.yml environment.yml
COPY api.py ./
COPY boot.sh ./
We haven’t yet created and discussed the file “boot.sh”. We will create and discuss this in a moment.
The following lines in the Dockerfile run two commands then set an environment path to the Conda environment which will be set up based on our environment.yml file and finally exposes port 5000, meaning we will be able to interact with our Flask routes through that port.
RUN conda env create -f environment.yml
RUN echo "source activate your-environment-name" > ~/.bashrc
ENV PATH /opt/conda/envs/your-environment-name/bin:$PATH
EXPOSE 5000
The first run command creates a Conda environment as specified in the environment.yml file. The second run command activates the created Conda environment.
However, there are a few lines that we have been ignoring thus far. The last line in Dockerfile is an entry point. Meaning this will be run after the image is created, furthermore, this will be executed from a non-privileged user.
The boot.sh file
In the entry point statement, we reference a file called booth.sh. The file contains the following.
#!/bin/sh
exec gunicorn -b :5000 --access-logfile - --error-logfile - api:app
This file is a script that executes one command. This command starts the Gunicorn web server, binds port 5000 to 80, and specifies that the Flask application to be run is in file api.
Alternatively, we might drop this part entirely and just run the Flask web server by executing the api.py file. However, as Flask documentation states it, they are not recommending using the Flask built-in web server in production. Therefore, we want to run a production-ready web server like Gunicorn.
To make this work, you need to create a file called boot.sh copy the above lines of code and save the file in the root folder, in my case this would be inside the folder ‘docker_conda_template’.
Finally, there are two remaining lines in the Dockerfile to explain:
COPY boot.sh ./
RUN chmod +x boot.sh
The first line simply copies the boot.sh file into the docker image in the root folder which would be your working directory as specified in the Dockerfile. The second line runs a command which modifies the boot.sh file so it can be recognized as an executable file.
Build and run your Docker image
To sum up now you should have the following folder/file structure.
docker_conda_template
--api.py
--boot.sh
--environment.yml
--Dockerfile
In order to build the docker image, you need to execute the following command.
docker build -t your-image-name:latest .
Start the Docker image by running the following command.
docker run –name your-image-name -p 80:5000 –rm your-container-name:latest
Open a browser and visit localhost or localhost/test and you should get a response as defined in your api.py file.
Interested in using Conda and Python for Machine Learning, then head over to my course Introduction to Machine Learning End-to-End. When you finish this course you will have a complete working code, which you can use for other projects!