Hello I'm new to prefect and started using it toda...
# prefect-community
m
Hello I'm new to prefect and started using it today its totally awesome piece but I have a problem with it. I have a script that is importing custom python modules from a few folders, when started as a "new" flow it runs fine but when i want to rerun it over the web ui its shows import errors for the custom imports. How do I get the folders into the environment ? Thanks upfront for any help.
1
j
Hi @Max Ernst your modules need to be importable off of the PYTHONPATH wherever your flows are being executed. Generally users perform this by using something like Docker storage to package the files with it and add their location to the path https://docs.prefect.io/api/latest/environments/storage.html#docker If using the Local agent you can set an
--import-path
on the start to specify directly where the files are if that fits your workflow
m
First, thanks for answering. Can I do that too with a s3 storage ?
j
If your flow is stored in S3 and you are using the Local agent then yes you can either add these modules to the PYTHONPATH globally or set
--import-path
on the agent start. Also optionally if you want to go the Docker route you can still store your flow in S3 and manually make your own Docker base image for executing the flow with your dependencies https://docs.prefect.io/orchestration/execution/storage_options.html#non-docker-storage-for-containerized-environments
m
Oky I got it now and fixed the problem with your help thank you a lot u saved me a lot a tinkering