storage. I have a simple flow and a deployment for that flow. I'm using
storage=S3(bucket_path="...")
It correctly uploads everything in the folder where I run the python file with my deployment (I can ignore some files using .prefectignore file). However, it uploads everything to the bucket. I can use the
path
argument of the deployment to
myflow
so files are uploaded inside that prefix in my bucket. The problem is that when I run a flow, the agent does not find the flow and it crashes.
Am I doing something wrong? Thank you!
✅ 1
r
redsquare
04/10/2023, 11:09 AM
can you show your deployment
n
Nelson Griffiths
04/10/2023, 11:26 AM
This may be the issue that was fixed in 2.10.2
j
jcozar
04/10/2023, 3:00 PM
Thank you for your responses! I'm a bit confused, should I use
prefect.filesystems.S3
or
prefect_aws.S3Bucket
?
jcozar
04/10/2023, 3:00 PM
With the second option it seems it worked
jcozar
04/10/2023, 11:22 PM
With
prefect_aws
you can select the bucket folder where you want to store the results/flows, it worked like charm 🙌
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.