Felix Vemmer

    Felix Vemmer

    1 year ago
    Hi everyone, I am getting the following error, when trying to write a pandas dataframe into google Cloud Storage:
    Unexpected error: TypeError("__init__() got an unexpected keyword argument 'client_options'")
    Traceback (most recent call last):
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/engine/runner.py", line 48, in inner
        new_state = method(self, state, *args, **kwargs)
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/engine/task_runner.py", line 891, in get_task_run_state
        result = self.result.write(value, **formatting_kwargs)
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/engine/results/gcs_result.py", line 77, in write
        self.gcs_bucket.blob(new.location).upload_from_string(binary_data)
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/engine/results/gcs_result.py", line 41, in gcs_bucket
        client = get_storage_client()
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/utilities/gcp.py", line 53, in get_storage_client
        return get_google_client(storage, credentials=credentials, project=project)
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/prefect/utilities/gcp.py", line 31, in get_google_client
        client = Client(project=project, credentials=credentials)
      File "/Users/felixvemmer/.pyenv/versions/3.8.6/envs/automation_beast/lib/python3.8/site-packages/google/cloud/storage/client.py", line 122, in __init__
        super(Client, self).__init__(
    TypeError: __init__() got an unexpected keyword argument 'client_options'
    I am running a task thats returning a
    pd.DataFrame
    which I am trying to store into Google Cloud Storage:
    pandas_serializer = PandasSerializer(
        file_type='csv'
    )
    
    gcs_result = GCSResult(
        bucket='tripliq-data-lake',
        serializer=pandas_serializer,
        location=f'linkedin_top_posts/{datetime.datetime.now().strftime("%Y%m%d-%H%M%S")}_linkedin_post_likes.csv'
    )
    
    like_linkedin_feed = LikeLinkedInFeed(
        result=gcs_result
    )
    I am not understanding the source code too well, but I think it’s referring to this line in
    site-packages/google/cloud/storage/client.py
    def __init__(
    self,
    project=_marker,
    credentials=None,
    _http=None,
    client_info=None,
    client_options=None,
    ):
    Any help is very much appreciated!
    j

    josh

    1 year ago
    Hi @Felix Vemmer could you open this as an issue on the Prefect repo with a reproducible example? Will be easier to triage there 🙂