https://prefect.io logo
Join Slack
Powered by
Hello everyone! I am trying to run a flow via the ...
# prefect-community
m
Hello everyone! I am trying to run a flow via the UI, which I stored in a Docker image. The docker image itself builds with no issue, as shown in the code and image bellow.
Copy code
storage = Docker(image_name="flows-storage", image_tag="v1.0",local_image=True,dockerfile="DockerFile-PrefectDependencies")
storage.build()
test_flow.storage=storage
test_flow.register(build=False,labels=['docker-execs-test'])
But when I start a docker agent and run the flow, the run is permanently in Submitted state. Any idea why this is?
👀 1
d
Hi @Manuel Mourato, Are there any logs from the flow run stuck in the submitted state? It’s possible your agent is having trouble pulling your image from wherever it lives (potentially due to a permissions issue).
m
Hello @Dylan This is the only log I see in the UI:
Copy code
May 21st 2020 at 8:37:26pm | agent
INFO 
Submitted for execution: Container ID: 2195b5fba5fdfe7aebd42cc3d2b24dfd388b593a7906696c158c14adffe333a5
This what appeared in the docker agent at the time of the submission:
Copy code
[2020-05-21 19:37:26,644] DEBUG - agent | Docker container 2195b5fba5fdfe7aebd42cc3d2b24dfd388b593a7906696c158c14adffe333a5 started
[2020-05-21 19:37:26,666] DEBUG - agent | Completed flow run submission (id: b5dc6bf7-c17f-4b64-aa00-ef75e92eddb1)
[2020-05-21 19:37:26,667] DEBUG - agent | No flow runs found
[2020-05-21 19:37:26,668] DEBUG - agent | Next query for flow runs in 1.0 seconds
[2020-05-21 19:37:27,675] DEBUG - agent | Querying for flow runs
....
The image is is listed as any other:
Copy code
REPOSITORY                             TAG                                IMAGE ID            CREATED             SIZE
flows-storage                          v1.0                               4fbaeb689778        2 hours ago         2.26GB
Is there any other place I can check for logs?
d
@Manuel Mourato I would go directly to your container for logs and see if there’s anything different
If you’d be willing to share your flow code I’m happy to take a look at that as well
m
When I run docker ps or ps -a I don't see a container with the ID mentioned above
Copy code
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS              PORTS                            NAMES
44b17ca9857e        prefecthq/ui:0.10.7                      "/intercept.sh"          2 weeks ago         Up 2 weeks          80/tcp, 0.0.0.0:8080->8080/tcp   tmp_ui_1
68bc69e6109c        prefecthq/server:0.10.7                  "python src/prefect_…"   2 weeks ago         Up 2 weeks                                           tmp_scheduler_1
2aa5452f37ab        prefecthq/apollo:0.10.7                  "npm run serve"          2 weeks ago         Up 2 weeks          0.0.0.0:4200->4200/tcp           tmp_apollo_1
99fb78a30f84        prefecthq/server:0.10.7                  "bash -c 'prefect-se…"   2 weeks ago         Up 2 weeks          0.0.0.0:4201->4201/tcp           tmp_graphql_1
fff0e7c91174        hasura/graphql-engine:v1.1.0             "graphql-engine serve"   2 weeks ago         Up 2 weeks          0.0.0.0:3000->3000/tcp           tmp_hasura_1
70baeada2c2e        postgres:11                              "docker-entrypoint.s…"   2 weeks ago         Up 2 weeks          0.0.0.0:5432->5432/tcp           tmp_postgres_1
It doesn't seem to have created it from the mentioned image
Let me just adjust the code a bit and I will show you 🙂
Okay @Dylan, so after doing some tests I saw that the issue is in the Dockerfile that I am using. If I use a dockerfile like this, it works:
Copy code
FROM prefecthq/prefect:0.10.7-python3.6
ADD local_folder_content /root/content
However , if I use a dockerfile like this, despite it building the image correctly , it causes the issue I mentioned before:
Copy code
FROM ubuntu:18.04
RUN apt-get update && apt-get install -y python3.6 python3.6-dev python3-pip

RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN ln -s /usr/bin/python3.6 /usr/bin/python

RUN pip install --upgrade pip
RUN pip install --upgrade pip setuptools

ENV LC_ALL C.UTF-8
ADD local_folder_content /root/content

RUN pip install prefect==0.10.7
I can try and see what the prefecthq image contains that I am missing
@Dylan, I decided to adapt my dockerfile slightly to use prefecthq as a base image and it worked. Some of the code I ommitted from the above dockerfile was ubuntu 18.04 specific, and I had to adjust it to work with Debian buster Thank you for you time 🙂
j
Thanks for the update @Manuel Mourato 🙂
d
Glad you got it working!
Open in Slack
PreviousNext
Powered by