Hi Everyone, I tried to use the S3 storage for my ...
# ask-communityv
Vamsi Reddy
11/05/2021, 2:51 PMHi Everyone, I tried to use the S3 storage for my flow. I am running into an error
Failed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'utils\'")\nThis may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')k
Kevin Kho
11/05/2021, 2:54 PM
Hey @Vamsi Reddy, this is because only the Flow file itself is uploaded. You would need utils to be importable in the execution environment of the Flow. What agent/run config are you using?
v
Vamsi Reddy
11/05/2021, 2:54 PMi am using the local run
k
Kevin Kho
11/05/2021, 2:55 PM
You can try specifying the
working_dirutils Copy code
flow.run_config = LocalRun(working_dir="/path/to/working-directory")v
Vamsi Reddy
11/05/2021, 2:56 PMthank you i will give it a try
Vamsi Reddy
11/05/2021, 3:07 PMso i tried setting the working directory but i get the same error
Copy code
flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=4)
flow.register(project_name='SIE_Automation')
flow.storage.build()
flow.run_config = LocalRun(working_dir='/Users/vamsi/Desktop/Automation/')
flow.run(parameters={"config": default_config})Vamsi Reddy
11/05/2021, 3:08 PMwhen i run it locally it works and registers the flow. but i cannot trigger it from prefect cloud
Vamsi Reddy
11/05/2021, 3:08 PMthe utils folder is within the Automation project
Vamsi Reddy
11/05/2021, 3:12 PMhere’s my project structure:
a
Anna Geller
11/05/2021, 3:17 PM
perhaps it makes sense to make those custom modules to an installable package? You could then install it in the local agent environment + the environment from which you register your flows.
In this thread there is an example of how you could set this up: https://prefect-community.slack.com/archives/C014Z8DPDSR/p1636039765276000?thread_ts=1636037972.269500&cid=C014Z8DPDSR
👍 1
3 Views