Juanes Grimaldos
07/25/2024, 3:05 AM# Use an official Python runtime as a parent image
FROM python:3.10-slim
# Add our requirements.txt file to the image and install dependencies
COPY requirements.txt .
COPY src/ /src/
# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
WORKDIR /src
EXPOSE 4200
# Run our flow script when the container starts
CMD ["python", "workflow.py"]
and I want to run a code that end like this:
@flow
def main_flow(
mlflow_path: str = "sqlite:///mlflow/mlflow.db",
experiment: str = "random-forest",
) -> None:
"""The main training pipeline"""
datasets = load_data()
best_hyper_params = train_model(datasets)
logging.info(f"Best hyperparameters: {best_hyper_params}")
register_model(mlflow_path, experiment)
if __name__ == "__main__":
main_flow.serve(
name="src",
cron="*/2 * * * *",
tags=["mlops", "tracking"],
description="keep track on the model performance",
version="0.1.0",
)
Marvin
07/25/2024, 3:05 AMDockerfile
# Use an official Python runtime as a parent image
FROM python:3.10-slim
# Add our requirements.txt file to the image and install dependencies
COPY requirements.txt .
COPY src/ /src/
RUN pip install --no-cache-dir -r requirements.txt
# Set the working directory
WORKDIR /src
EXPOSE 4200
# Set the Prefect API key and URL as environment variables
ENV PREFECT_API_URL=YOUR_PREFECT_API_URL
ENV PREFECT_API_KEY=YOUR_PREFECT_API_KEY
# Run our flow script when the container starts
CMD ["python", "workflow.py"]
Replace YOUR_PREFECT_API_URL
and YOUR_PREFECT_API_KEY
with your actual Prefect Cloud API URL and API key.
Next, build your Docker image:
bash
docker build -t my-prefect-project .
Finally, run the Docker container with Prefect settings:
bash
docker run -e PREFECT_API_URL=YOUR_PREFECT_API_URL -e PREFECT_API_KEY=YOUR_PREFECT_API_KEY my-prefect-project
For detailed steps, refer to these resources:
1. Dockerizing Prefect Guide,
2. Prefect Deployment Guide.Juanes Grimaldos
07/25/2024, 3:05 AM# Use an official Python runtime as a parent image
FROM python:3.10-slim
# Add our requirements.txt file to the image and install dependencies
COPY requirements.txt .
COPY src/ /src/
# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
WORKDIR /src
EXPOSE 4200
# Run our flow script when the container starts
CMD ["python", "workflow.py"]
and I want to run a code that end like this:
@flow
def main_flow(
mlflow_path: str = "sqlite:///mlflow/mlflow.db",
experiment: str = "random-forest",
) -> None:
"""The main training pipeline"""
datasets = load_data()
best_hyper_params = train_model(datasets)
<http://logging.info|logging.info>(f"Best hyperparameters: {best_hyper_params}")
register_model(mlflow_path, experiment)
if __name__ == "__main__":
main_flow.serve(
name="src",
cron="*/2 * * * *",
tags=["mlops", "tracking"],
description="keep track on the model performance",
version="0.1.0",
)
Juanes Grimaldos
07/25/2024, 3:08 AMMarvin
07/25/2024, 3:09 AMDockerfile
# Use an official Python runtime as a parent image
FROM python:3.10-slim
# Set working directory
WORKDIR /src
# Add our requirements.txt file to the image and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy the src directory to the Docker image
COPY src/ /src/
EXPOSE 4200
# Run our flow script when the container starts
CMD ["python", "workflow.py"]
2. Create Your Prefect Flow:
Ensure your workflow.py
looks like this:
python
from prefect import flow
import logging
@flow
def main_flow(
mlflow_path: str = "sqlite:///mlflow/mlflow.db",
experiment: str = "random-forest",
) -> None:
"""The main training pipeline"""
datasets = load_data()
best_hyper_params = train_model(datasets)
<http://logging.info|logging.info>(f"Best hyperparameters: {best_hyper_params}")
register_model(mlflow_path, experiment)
if __name__ == "__main__":
main_flow.serve(
name="src",
cron="*/2 * * * *",
tags=["mlops", "tracking"],
description="keep track on the model performance",
version="0.1.0",
)
3. Build and Run Your Docker Image:
After creating the necessary files, you can build and run your Docker container locally.
### Build Your Docker Image
bash
docker build -t my-prefect-project .
### Run Your Docker Container Locally
Since you want to run it locally and ensure Prefect is accessible, you can run:
bash
docker run -e PREFECT_API_URL=<http://host.docker.internal:4200/api> -e PREFECT_API_KEY=YOUR_PREFECT_API_KEY -p 4200:4200 my-prefect-project
Replace YOUR_PREFECT_API_KEY
with your actual Prefect API key if you are using Prefect Cloud. If using Prefect Server locally, ensure your PREFECT_API_URL is correctly pointing to your local server instance.
By setting the environment variables and exposing the necessary port, you'll ensure that the Prefect server and API are accessible from within the Docker container.
For more details on local deployments, see:
- Prefect Deploy
- Prefect Docker Guide