https://prefect.io logo
Title
r

Russell Brooks

12/01/2022, 6:09 AM
The RemoteFileSystem example for MinIO doesn’t work when I try to build deployment with it as storage. I suspect their also needs to be a folder at the end of basepath or it will complain about malformed xml. Also once I add my folder I get past the xml error, but it tells me the specified bucket does not exist. How do I find out what’s going on? Been blocked on this for some hours now. As context the same MinIO bucket and folder worked in Prefect 1.0 . In Prefect 1.0 I used S3(bucket=S3_BUCKET, key=f”scripts/{flow_name}”, client_options=dict(endpoint_url=S3_ENDPOINT_URL, aws_access_key_id=KEY, aws_secret_access_key=SECRET_KEY). And this worked. Any ideas how to get that working in 2.0?
z

Zanie

12/01/2022, 6:11 AM
I think you might need to use the S3Bucket filesystem in
prefect-aws
Hm I do see we have an example with MinIO explicitly in the docs — sorry it’s not working for you.
If you open an issue we can investigate it.
r

Russell Brooks

12/01/2022, 6:20 AM
It’s not strictly S3. It is MinIO and I need to use an endpoint url. Not sure how to open an issue but if I did I would just copy paste what I’ve written above. Said another way, I had MinIO storage working in 1.0 and don’t know how to migrate it to 2.0
Anything blocking 1 -> 2 migration is a high priority for us, but we’ll need additional details.
r

Russell Brooks

12/01/2022, 6:46 AM
fix the documentation? Basepath needs a folder or it errors out. I cannot give much details as company policy is to not post code. Maybe start Should I be using RemoteFileSystem or AWS S3? If AWS how do I pass it a endpoint_url? Basically more examples using MinIO as storage would help
f

Faheem Khan

12/01/2022, 6:58 AM
@Russell Brooks you should create a bucket on minio first. with the same name as the one at the base path i.e. create a bucket on minio "prefect-flows" and base path "s3://prefect-flows/"