Hi there, I am struggling to run a flow on a Dock...
# ask-community
p
Hi there, I am struggling to run a flow on a Docker Container. I have set up a Prefect Server and a DockerAgent: Then try to run this flow:
Copy code
import prefect
from prefect import Flow, task
from prefect.run_configs import DockerRun
from prefect.storage import Docker

logger = prefect.context.get("logger")

@task
def task01():
    <http://logger.info|logger.info>("Task 01")

@task
def task02():
    <http://logger.info|logger.info>("Task 02")

@task
def task03():
    <http://logger.info|logger.info>("Task 03")


with Flow("Check Computers",
        storage=Docker(dockerfile="Dockerfile", image_name="check_computers", image_tag="latest"),
        run_config=DockerRun(image="check_computers:latest")) as flow:

    task01()
    task02()
    task03()

# Register the flow under the "tutorial" project
flow.register(project_name="CIS")
Then, in the logs I've got:
Failed to load and execute Flow's environment: FileNotFoundError(2, 'No such file or directory')
Does anyone know what I am missing?
a
Overall, it looks good, but there are some things you can make more explicit which may help solve the issue: 1) The logger cannot be defined globally, you should get it from the context in each task:
Copy code
import prefect
from prefect import Flow, task
from prefect.run_configs import DockerRun
from prefect.storage import Docker


@task
def task01():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("Task 01")


@task
def task02():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("Task 02")


@task
def task03():
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>("Task 03")
2) Specifying an explicit path to the Dockerfile may help as well. 3) Make sure your Server has Prefect version which is >= registration environment 4) Adding label to your agent may help to make it more explicit as well:
Copy code
run_config=DockerRun(image="check_computers:latest", labels=["docker"],),
5) Check if your flow uses some custom modules and if so, make sure those are copied into or installed within the image. If you tried all that and none of this helped, you can share your Dockerfile and the output of “prefect diagnostics” and I will try to reproduce.
p
Thanks @Anna Geller I'll apply your suggestions
👍 1
@Anna Geller This is the Dockerfile
Copy code
FROM python:3.8-slim

WORKDIR /app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE=true

COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir --upgrade pip \
    && pip install --no-cache-dir --prefer-binary -r /tmp/requirements.txt \
    && find / -name *.pyc -delete

# Copy all files
COPY . .
Do I need to add a CMD command to run the flow? ex:
CMD _[_ "python"_,_  "flow.py" _]_
Actually, thinking better there is no sense to run flow.py inside the container because it will just register it again 😅
a
no, you don’t have to - Prefect will replace it with command to start a flow run. Can you try using Prefect base image instead?
Also, you can register using the CLI “prefect register --project xxx -p flow.py” so you don’t need to have it in the flow file
1
p
Ok, I'll But I have other flows that needs specific images, that's why I am not using the prefect base image
👍 1
Also, you can register using the CLI “prefect register --project xxx -p flow.py” so you don’t need to have it in the flow file
@Anna Geller, In that case, should I add a
flow.run()
in the flow.py?
a
no, you don’t need it, Prefect runs your flow automatically
1