Hi. I'm looking at Prefect 2.0. The documentation shows support for S3 for storage. By chance, d...

Joshua Allen

06/15/2022, 7:18 PM

Hi. I'm looking at Prefect 2.0. The documentation shows support for S3 for storage. By chance, does this S3 support extend to a self-hosted S3-compliant service, such as MinIO? Otherwise, what are the options for a self-hosted environment? The documentation states that remote storage is required when running in k8s or docker. Would the File Storage on a Remote File System option, such as SMB or NFS, work?

Kevin Kho

06/15/2022, 7:36 PM

Not yet. It’s waiting for this . I don’t think NFS will work as it’s not considered Remote storage. I need to double check on that.

Kevin Kho

06/15/2022, 7:44 PM

I think I was wrong. You can use File storage, just not Local

Joshua Allen

06/15/2022, 8:08 PM

thank you!

davzucky

06/15/2022, 11:15 PM

please check here. You can do that using fsspec https://discourse.prefect.io/t/when-should-i-use-filestorageblock-vs-specific-cloud-pla[…]dstorageblock-or-azureblobstorageblock/924/2?u=anna_geller

upvote 1

davzucky

06/15/2022, 11:16 PM

At the point you can setup the storage with fsspec, they are a lot of underlines available. Just need to find the right the setup them

🙏 1

4 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.