Scott Zelenka
03/27/2020, 2:05 PMS3ResultHandler doesn't expose all of the boto3.client arguments, which prevents us from re-using this to connect to non-AWS S3 compliant Object Stores (such as MinIO).
I created a wrapper around S3ResultHandler which simply allows the user to specify more of the parameters at initialization, and then forwards those parameters off to boto3.client when it's initialized.
Rather than creating another Result Handler as a separate Python package, would it make more sense to modify the S3ResultHandler in Prefect Core?
Or does it make more sense to have a dedicated Result Handler specific to MinIO, and use their Python wrapper around boto3?josh
03/27/2020, 2:39 PMs3 for the boto3.client?Scott Zelenka
03/27/2020, 2:48 PMendpoint_url that's not AWS. We also specify some extra config parameters and region_name information that's specific to MinIO instance:
s3_client = boto3.client(
service_name="s3",
region_name=self.region_name,
api_version=self.api_version,
use_ssl=self.use_ssl,
verify=self.verify,
endpoint_url=self.endpoint_url,
aws_access_key_id=aws_access_key,
aws_secret_access_key=aws_secret_access_key,
aws_session_token=None,
config=Config(**self.config)
)Scott Zelenka
03/27/2020, 2:48 PMinitialize_client method of S3ResultHandler to pass through those extra parameters to boto3josh
03/27/2020, 2:49 PMAlex Cano
03/27/2020, 3:24 PMScott Zelenka
04/01/2020, 7:30 PMjosh
04/01/2020, 7:31 PMjosh
04/01/2020, 7:31 PM