Scarlett King
10/05/2021, 2:53 PMKevin Kho
10/05/2021, 2:54 PMScarlett King
10/05/2021, 2:55 PMKevin Kho
10/05/2021, 3:03 PMScarlett King
10/05/2021, 3:06 PMKevin Kho
10/05/2021, 3:07 PMScarlett King
10/05/2021, 3:13 PMKevin Kho
10/05/2021, 3:15 PMprefect agent … start --no-hostname-label
?Scarlett King
10/05/2021, 3:18 PMKevin Kho
10/05/2021, 4:50 PMScarlett King
10/05/2021, 5:17 PMKevin Kho
10/05/2021, 5:22 PMScarlett King
10/05/2021, 6:04 PMKevin Kho
10/05/2021, 8:35 PMflow.run()
?Scarlett King
10/06/2021, 7:56 AMflow.run()
works fine. On this flow, I’m using Path from pathlib to handle file path and such. I would have a task that take filepath as an input and would pass filepath= ROOT_DIR / ‘filename.csv’
where ROOT_DIR = Path(__file__).resolve().parent
. So I removed these and set the file path as 1 string, I believe this only occurs when I pass the value to a task function. When I run this inside a task it works fine with no error.
def save_file(df):
file_path = ROOT_DIR / ‘file_name.csv’
df.to_csv(file_path)
def save_file(df, file_path):
df.to_csv(file_path)
with Flow(‘abc’, storage=azure_store, run_configs=LocalRun()) as flow:
df = query_db(sql_query)
save = save_file(df, file_name=ROOT_DIR / ‘file_name.csv’)
kubernetes_run = KubernetesRun(image=os.environ.get(“IMAGE_URL”), job_template_path=os.environ.get(“JOB_TEMPLATE_PATH”, ROOT_DIR / “job_template.yaml”))
ROOT_DIR / “job_template.yaml”
with ”./job_template.yaml”
flow.run()
as it can’t find the job_template.yaml filestr(ROOT_DIR / “job_template.yaml”)
Kevin Kho
10/06/2021, 2:32 PMScarlett King
10/06/2021, 3:32 PM