Hi all, I'm just getting started with Prefect and Docker as well. Basically, I'm trying to find out how I go from executing flow runs using the Local agent to using the Docker agent.
I have a registered flow that's stored on Github. This runs fine using a Local agent in my local environment.
Attempting to run the flow with a Docker agent fails with
ModuleNotFoundError
. To make this work will I need to create a docker image that includes all the dependencies? Also, this means that I will have to use another option for storage?
k
Kevin Kho
09/09/2021, 3:21 PM
Hey @Alex, yes to docker image and we have Docker storage, but I have a walkthrough and example ready for you!
Check this and the code here
Kevin Kho
09/09/2021, 3:22 PM
Feedback would be appreciated on the blog draft btw. GithubStorage will only pull the flow and not the dependencies so you need them as a Python package in the Docker image.
a
Alex
09/09/2021, 3:56 PM
Thanks @Kevin Kho! That's what I was assuming. I'm not sure that the link to the medium article is working...
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.