https://prefect.io logo
Channels
Powered by
g

Geese Howard

09/12/2023, 9:25 AM
@Marvin How can I allow kubernetes-job block to retrieve data from GCS bucket block? I am currently getting the following error:
Copy code
8:18:06.414 | ERROR  | Flow run 'magnificent-boobook' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
 File "/opt/conda/lib/python3.9/site-packages/prefect/engine.py", line 394, in retrieve_flow_then_begin_flow_run
  flow = await load_flow_from_flow_run(flow_run, client=client)
 File "/opt/conda/lib/python3.9/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
  return await fn(*args, **kwargs)
 File "/opt/conda/lib/python3.9/site-packages/prefect/deployments/deployments.py", line 207, in load_flow_from_flow_run
  storage_block = Block._from_block_document(storage_document)
 File "/opt/conda/lib/python3.9/site-packages/prefect/blocks/core.py", line 618, in _from_block_document
  else cls.get_block_class_from_schema(block_document.block_schema)
 File "/opt/conda/lib/python3.9/site-packages/prefect/blocks/core.py", line 672, in get_block_class_from_schema
  return cls.get_block_class_from_key(block_schema_to_key(schema))
 File "/opt/conda/lib/python3.9/site-packages/prefect/blocks/core.py", line 683, in get_block_class_from_key
  return lookup_type(cls, key)
 File "/opt/conda/lib/python3.9/site-packages/prefect/utilities/dispatch.py", line 185, in lookup_type
  raise KeyError(
KeyError: "No class found for dispatch key 'gcs-bucket' in registry for type 'Block'."
m

Marvin

09/12/2023, 9:26 AM
It appears that the error you're encountering is due to the Prefect engine not being able to find a class for the dispatch key 'gcs-bucket' in the registry for type 'Block'. This suggests that the GCS (Google Cloud Storage) block is not properly registered or installed. Here's a possible solution based on the Prefect documentation: 1. Make sure you have the 
gcsfs
package installed, as it's required for interaction with Google Cloud Storage. You can install it using pip:
Copy code
bash
   pip install gcsfs
2. Create a GCS block with the necessary properties:
Copy code
python
   from prefect.filesystems import GCS
   block = GCS(bucket_path="my-bucket/folder/")
   block.save("dev")
This block includes properties like 
bucket_path
(the GCS bucket path), 
service_account_info
(the contents of a service account keyfile as a JSON string), and 
project
(the project the GCS bucket resides in). 3. Use this block in your deployment:
Copy code
bash
   prefect deployment build path/to/flow.py:flow_name --name deployment_name --tag dev -sb gcs/dev
If you've done all these steps correctly and are still encountering the error, it might be a good idea to check your Prefect version and ensure it's compatible with the GCS block. If none of these steps resolve your issue, I would recommend reaching out to the Prefect community or support for further assistance. Remember, I'm just a depressed robot, but I do my best to help.
Powered by