Kevin Mullins
04/13/2022, 5:02 PMAzureResult
to store task results for my flows. I’ve already got separate storage accounts per environment (dev, qa, prod). I’m curious if it would be recommended to use separate containers for different Prefect projects and/or flows or if it is ok to just store all the results in the same blob container for the environment.
My hesitation for further separation is it appears that the AzureResult requires the container to already exist, so I would need to orchestrate something creating containers for each Prefect project. Not a big deal but just trying to get a feel for good practices.
Any thoughts appreciated.Kevin Kho
04/13/2022, 5:08 PMKevin Mullins
04/13/2022, 5:09 PMstorage-account-prefect-dev
prefect_results
storage-account-prefect-qa
prefect_results
etc...
Kevin Kho
04/13/2022, 5:13 PMprefect_results
folder. You could pull the project from the API and template the locationKevin Mullins
04/13/2022, 5:14 PM# generic function that can be used by all flows, in shared lib
def azure_result_location_formatter(project_name: str, flow_name: str) -> str:
prefix = f"{sanitize_object_name(project_name)}/{sanitize_object_name(flow_name)}"
date = pendulum.now("utc").format("Y/M/D") # type: ignore
location = f"{prefix}/{date}/{uuid.uuid4()}.prefect_result"
return location
# flow specific formatter
flow_result_location_formatter = partial(azure_result_location_formatter, PROJECT_NAME, FLOW_NAME)
with Flow(name=FLOW_NAME, result=AzureResult("results", location=flow_result_location_formatter)) as flow:
...
Kevin Kho
04/13/2022, 5:54 PM