Hi ! I have a flow which triggers multiple subflows. I want to run them on multiple workers since they don't depend on each other. But I want to retrieve the return statement of every subflow in my
main flow which is a basic
string.
Here's my current infra :
• prefect api server within a docker container
• postgres db within a docker container
• prefect-worker within a docker container (process work pool)
I have one process work pool with a deployment (which is the subflow I wanna run)
I run my main flow locally on my computer (for dev purposes). It runs correctly the subflows within the
prefect-worker container but when I use
await result.state.result(_fetch_=True, _retry_result_failure_=True, _raise_on_failure_=False)
, my
main flow run into
ValueError: Path /root/.prefect/storage/f2ffbc1d077445acac57a0be51101714 does not exist.
Here are my questions :
• Where is located that storage ? in the
prefect-worker container ?
• Can I avoid persistent storage ?
• Can I avoid setting up an S3 storage only for storing a simple string ?
Thank you.