https://prefect.io
Join Slack
Hi Everyone, I tried to use the S3 storage for my flow. I am running into an error `Failed to load a...
# ask-community
v
Hi Everyone, I tried to use the S3 storage for my flow. I am running into an error 
Failed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'utils\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I am able to run the flow locally but i get the error when i trigger it from the prefect cloud.
k
Hey @Vamsi Reddy, this is because only the Flow file itself is uploaded. You would need utils to be importable in the execution environment of the Flow. What agent/run config are you using?
v
i am using the local run
k
You can try specifying the 
working_dir
to point it to a place where you can import the 
utils
Copy code
flow.run_config = LocalRun(working_dir="/path/to/working-directory")
v
thank you i will give it a try
so i tried setting the working directory but i get the same error
Copy code
flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=4)
flow.register(project_name='SIE_Automation')
flow.storage.build()
flow.run_config = LocalRun(working_dir='/Users/vamsi/Desktop/Automation/')
flow.run(parameters={"config": default_config})
when i run it locally it works and registers the flow. but i cannot trigger it from prefect cloud
the utils folder is within the Automation project
here’s my project structure:
a
perhaps it makes sense to make those custom modules to an installable package? You could then install it in the local agent environment + the environment from which you register your flows. In this thread there is an example of how you could set this up: https://prefect-community.slack.com/archives/C014Z8DPDSR/p1636039765276000?thread_ts=1636037972.269500&cid=C014Z8DPDSR
👍 1
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by