Hey community, I have previously worked with Dockerstorage for all my flows but as my repo has grow...

Ben Muller

08/22/2021, 10:27 PM

Hey community, I have previously worked with Dockerstorage for all my flows but as my repo has grown it has slown down my CI process, so I am switching to S3 stroage. I have a docker image baked in with all the dependancies and files etc for the repo. I have set up my storage like so:

Copy code

S3(
            bucket="es-prefect-flows-staging",
            stored_as_script=True,
        )

Now after registering my flow I am getting an error after the flow is downloaded: in 🧵

✅ 1

Ben Muller

08/22/2021, 10:28 PM

Failed to load and execute Flow's environment: ImportError("cannot import name 'some_flow' from 'flows.tab.tasks' (/flows/tab/tasks.py)")

Ben Muller

08/22/2021, 10:29 PM

I built this image locally and the tasks file is located at

/flows/tab/tasks.py

wondering how you suggest to debug something like this?

Ben Muller

08/22/2021, 10:32 PM

maybe something to do with the

PYTHONPATH

but not sure how I can set this on the

S3

storage object?

Ben Muller

08/22/2021, 10:32 PM

Do I need to set it on the run config?

Ben Muller

08/22/2021, 10:46 PM

answered my own question 😆

Zanie

08/23/2021, 2:22 PM

😄 nice job

😆 1

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.