Dhruv Laad
12/24/2024, 6:38 PMAzureBlobStorageContainer
block. I have the block registered on prefect as shown in the attached screenshot.
Referring to the code snipped there, I create block stores to use in my tasks and flows as follows:
from prefect_azure import AzureBlobStorageContainer, AzureBlobStorageCredentials
def prefect_block(block_name, base_folder=None):
try:
return AzureBlobStorageContainer.load(block_name)
except ValueError:
creds_block = AzureBlobStorageCredentials(connection_string=conn_str)
try:
creds_block = creds_block.load(name="creds_block")
except ValueError:
creds_block.save(name=f"creds_block")
creds_block = creds_block.load(name="creds_block")
azure_block = AzureBlobStorageContainer(container_name=container, credentials=creds_block, base_folder=base_folder)
azure_block.save(name=block_name)
return AzureBlobStorageContainer.load(block_name)
I then specify the necessary result store for my tasks like so:
RESULT_STORE = prefect_block("common-result-store", "common/data/result_store")
@task(result_storage=RESULT_STORE)
def foo():
...
@flow(result_storage=RESULT_STORE)
def bar():
...
For my flow I am using task_runner=DaskRunner
. On running my flow I get the following error in one of the tasks:
TypeError: Result storage configuration must be persisted server-side. Please call `.save()` on your block before passing it in.
alex
12/24/2024, 6:46 PMprefect
you have installed?Dhruv Laad
12/24/2024, 6:56 PMVersion: 3.1.9
API version: 0.8.4
Python version: 3.10.12
Git commit: e1fe7943
Built: Fri, Dec 20, 2024 4:33 PM
OS/Arch: linux/x86_64
Profile: local
Server type: server
Pydantic version: 2.9.2
Integrations:
prefect-dask: 0.3.2
prefect-azure: 0.4.2
alex
12/24/2024, 6:59 PM@task(result_storage="azure-blob-storage-container/common-result-store")
.Dhruv Laad
12/24/2024, 7:01 PMalex
12/24/2024, 7:04 PMDhruv Laad
12/24/2024, 7:08 PMalex
12/24/2024, 7:09 PMDhruv Laad
12/24/2024, 7:11 PM