Lana Dann12/02/2021, 9:20 PM
for storage? for context, i’m using gitlab storage for my flow and flows are in
but when the flow tries to run, it errors with
when the flow tries to access the storage.
ModuleNotFoundError: No module named 'lib'
Kevin Kho12/02/2021, 9:20 PM
Lana Dann12/02/2021, 9:21 PM
Lana Dann12/02/2021, 9:33 PM
Kevin Kho12/02/2021, 9:36 PM
and the agent starts a new ECS container with that image and then gets the Flow from storage and runs it on top
Lana Dann12/02/2021, 9:43 PM
container and then executes the flow. is that correct?
Kevin Kho12/02/2021, 9:48 PM
as a CLI command in the container to start the flow run.
prefect execute flow
is not a storage thing (unless you use Docker storage). The script based storages like Github or S3 just hold the flow code and that flow code is pulled down when the flow is executed. The
has to go on the execution environment (the container)
Lana Dann12/02/2021, 9:53 PM
file that i import in my flow? does that also get copied over?
Kevin Kho12/02/2021, 9:55 PM
did not support deep copying of Python modules (until very recently). With the
development this experience might be improved, but that is definitely a Prefect 2.0 thing. So yes, only the flow file is copied over by default and the dependencies need to be added to the execution environment (through the container)