Brad Clark
11/04/2022, 4:14 PMprefect_gcp.cloud_storage
as described here https://prefecthq.github.io/prefect-gcp/. I'm tryin to provide some direction to our data engineersalex
11/04/2022, 5:49 PMGCS
block can read and write file and full directories from GCS but doesn’t offer many configuration options. The tasks in prefect_gcp.cloud_storage
offer more configuration options, but don’t support reading and writing full directories out of the box. The amount of configuration that you need would determine which to reach for. Long term, you can expect to see a lot of the GCS functionality centralize in the GcsBucket
block which resides in prefect_gcp.cloud_storage
. Our goal is to make that the go to solution for interacting with GCS buckets.