Skip to content

Dockerize the PA pipeline #463

Description

@pauldhein

Overview

We need to create a docker container that can run the full translation process from Fortran source to GrFN. As a first step in that process we need to create a docker container that will handle running the PA pipeline. @hlim1 I am assigning this task to you but feel free to ask me and @pratikbhd for assistance as needed.

@skdebray I am adding you to this issue so that you will be kept up-to-date about our teams progress.

Task Goal

Create a minimal docker container that can run the program analysis pipeline.

Implementation Details

  • Make all changes needed for this task on the dssat_pet branch
  • Locate the Dockerfile for the container in: delphi/translators/for2py
  • Use the Dockerfile provided below as a stub for the new Dockerfile
    • Remove dependencies that are unnecessary for for2py
    • Change the last few lines to use delphi (for now) and set the WORKDIR to be ready for script execution
  • Have the docker container execute the script: scripts/f2grfn_standalone.py
  • Use PETPT.for as the test argument while making the container
  • When the docker container is run via the CMD it should generate an _AIR.JSON file for the input fortran module and return that to the directory on the on the source machine where the docker container was launched from

Dockerfile Stub

FROM        ubuntu:19.10
MAINTAINER  Paul D. Hein <pauldhein@email.arizona.edu>
CMD         bash

# ==============================================================================
# INSTALL SOFTWARE VIA THE UBUNTU PACKAGE MANAGER
# ==============================================================================
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get -y --no-install-recommends install \
  git \
  curl \
  build-essential \
  gcc \
  pkg-config \
  python3 \
  python3-dev \
  python3-pip \
  openjdk-8-jdk \
  antlr4 \
  graphviz \
  libgraphviz-dev \
  doxygen

RUN git config --global user.email "pauldhein@email.arizona.edu"
RUN git config --global user.name "Paul Hein"
# ==============================================================================

# ==============================================================================
# INSTALL PYTHON PACKAGES VIA PIP
# NOTE: Packages are installed in stages to avoid memory errors on DockerHub
# ==============================================================================
# Upgrading setuptools and installing wheel
RUN pip3 install --upgrade setuptools
RUN pip3 install wheel

# Common-use and testing packages
RUN pip3 install \
  antlr4-python3-runtime==4.8 \
  pytest==5.4.1 \
  pytest-cov==2.8.1 \
  Pygments==2.3.1 \
  tqdm==4.29.0

# Adding PyTorch separately due to memory constraints
RUN pip3 install \
  torch==1.4.0+cpu \
  -f https://download.pytorch.org/whl/torch_stable.html

# Scientific computing packages
RUN pip3 install \
  networkx==2.4 \
  numpy==1.18.2 \
  pandas==1.0.3 \
  sympy==1.5.1 \
  scikit_learn==0.22.2.post1 \
  SALib==1.3.8 \
  nltk==3.4.5 \
  torchtext==0.5.0

# Python Visualization libraries
RUN pip3 install \
  pygraphviz==1.5 \
  matplotlib==3.2.1 \
  plotly==4.5.4 \
  seaborn==0.10.0

# Web framework packages
RUN pip3 install \
  Flask==1.1.1 \
  flask_codemirror==1.1 \
  flask_wtf==0.14.3 \
  WTForms==2.2.1

# Unassigned other stuff
RUN pip3 install SPARQLWrapper==1.8.5
# ==============================================================================

# ==============================================================================
# SETUP THE AUTOMATES REPOSITORY AND ENVIRONMENT
# ==============================================================================
RUN git clone https://github.com/ml4ai/automates
ENV PYTHONPATH="/automates/src:$PYTHONPATH"
WORKDIR /automates
# ==============================================================================

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions