Moritz Rocholl
08/21/2024, 6:27 PMMarvin
08/21/2024, 6:28 PM# Use the official Python base image
FROM python:3.9-slim
# Set the working directory
WORKDIR /app
# Copy the requirements file
COPY requirements.txt .
# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application code
COPY . .
# Set the entrypoint for the container
ENTRYPOINT ["python", "my_flow_script.py"]
Make sure your flow script is in the same directory and is named my_flow_script.py
. Also, create a requirements.txt
file containing:
requirements.txt:
prefect
httpx
# Add any other dependencies required by your flow
Here's a basic example of a Prefect flow script you can use:
my_flow_script.py:
python
from prefect import flow
import httpx
@flow(log_prints=True)
def get_repo_info(repo_name: str = "PrefectHQ/prefect"):
url = f"<https://api.github.com/repos/{repo_name}>"
response = httpx.get(url)
response.raise_for_status()
repo = response.json()
print(f"{repo_name} repository statistics 🎉:")
print(f"Stars 🌟 : {repo['stargazers_count']}")
print(f"Forks 🍴 : {repo['forks_count']}")
print(f"Open issues 🐞 : {repo['open_issues_count']}")
if __name__ == "__main__":
get_repo_info()
To build the Docker image, run in your terminal:
docker build -t my_prefect_flow .
For more detailed information, refer to the Prefect documentation on Docker.Moritz Rocholl
08/21/2024, 6:31 PMMoritz Rocholl
08/21/2024, 6:32 PMMarvin
08/21/2024, 6:32 PMget_repo_info.deploy()
, you'll need to ensure that the Docker image you build can execute Prefect flows and is set up correctly for the deployment.
Here's an updated approach:
1. Custom Dockerfile:
This Dockerfile includes all necessary steps to create an environment compatible with Prefect.
Dockerfile:
# Use the official Python base image
FROM python:3.9-slim
# Set the working directory
WORKDIR /app
# Copy the requirements file
COPY requirements.txt .
# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application code
COPY . .
# Set the entrypoint for the container
ENTRYPOINT ["python", "-m", "prefect.engine"]
2. requirements.txt:
Ensure the required dependencies are listed here.
requirements.txt:
prefect
httpx
# Add any other dependencies required by your flow
3. Flow Script (my_flow_script.py):
Update your flow script to include the deployment part.
my_flow_script.py:
python
from prefect import flow
import httpx
@flow(log_prints=True)
def get_repo_info(repo_name: str = "PrefectHQ/prefect"):
url = f"<https://api.github.com/repos/{repo_name}>"
response = httpx.get(url)
response.raise_for_status()
repo = response.json()
print(f"{repo_name} repository statistics 🎉:")
print(f"Stars 🌟 : {repo['stargazers_count']}")
print(f"Forks 🍴 : {repo['forks_count']}")
print(f"Open issues 🐞 : {repo['open_issues_count']}")
if __name__ == "__main__":
get_repo_info.deploy(
name="my-first-deployment",
work_pool_name="my-docker-pool",
image="my-first-deployment-image:tutorial",
push=False # Set to True if you need to push to a remote registry
)
4. Build the Docker Image:
Use the docker build
command to create your Docker image.
docker build -t my-first-deployment-image:tutorial .
This setup ensures that your custom Dockerfile is compatible with get_repo_info.deploy()
. By deploying your flow using Flow.deploy()
, Prefect can handle running the flow inside a Docker container with the specified image. If Prefect is running a Docker worker on your local machine, you can set push=False
. Otherwise, ensure to push your image to a registry accessible by Prefect workers.
For more detailed and updated instructions, refer to the Prefect documentation on Docker and Prefect flow deployment.