Marius Vollmer
05/13/2023, 12:05 PMpath
and entrypoint
so that the central template becomes unique deployments in remote storage. Unfortunately, I am currently failing with FileNotFoundError.
Example deployment and error message can be found in the thread.
What am I missing/doing wrong? Thanks in advance for any hints!Marius Vollmer
05/13/2023, 12:07 PMdef deploy_orchestrator_flow(
flow_dict
, deploy_store_name
, env):
storage = Azure.load(deploy_store_name)
deployment = Deployment.build_from_flow(
flow=orchestrator_flow,
name=flow_dict['orchestrator_flow_name']+'_deployment',
description=flow_dict['orchestrator_flow_description'],
version=f"{env}",
work_queue_name=flow_dict['work_queue'],
storage=storage,
path="/flows/dbt/kafka_order", # "/flows/dbt/kafka_order",
entrypoint="flow_configs_stg_to_cdw_template.py:orchestrator_flow",
tags=flow_dict['tags'].split(),
schedule=(CronSchedule(cron=flow_dict['schedule_cron'], timezone="Europe/Berlin")),
is_schedule_active=True,
infrastructure=kubernetes_job_block,
infra_overrides={
}, "env": {
"EXTRA_PIP_PACKAGES": "adlfs==2023.1.0 snowflake-connector-python python-dotenv==0.21.0 prefect==2.10.4 prefect-dbt==0.3.1 prefect-snowflake==0.26.1 dbt-snowflake==1.4.1 pygit2==1.11.1",
}, "namespace": "snowflake-stg"
}
},
)
deployment.apply()
The file flow_configs_stg_to_cdw_template.py is located in the remote storage below the path directory.
Error message of the flow run:
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "<frozen importlib._bootstrap_external>", line 879, in exec_module.
File "<frozen importlib._bootstrap_external>", line 1016, in get_code
File "<frozen importlib._bootstrap_external>", line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/opt/prefect/flow_configs_stg_to_cdw_template.py'.