Scarlett King
10/05/2021, 2:53 PMKevin Kho
Scarlett King
10/05/2021, 2:55 PMScarlett King
10/05/2021, 2:55 PMScarlett King
10/05/2021, 2:57 PMKevin Kho
Scarlett King
10/05/2021, 3:06 PMKevin Kho
Scarlett King
10/05/2021, 3:13 PMKevin Kho
prefect agent … start --no-hostname-label
?Scarlett King
10/05/2021, 3:18 PMScarlett King
10/05/2021, 4:07 PMScarlett King
10/05/2021, 4:33 PMScarlett King
10/05/2021, 4:47 PMKevin Kho
Kevin Kho
Kevin Kho
Scarlett King
10/05/2021, 5:17 PMScarlett King
10/05/2021, 5:20 PMKevin Kho
Scarlett King
10/05/2021, 6:04 PMScarlett King
10/05/2021, 6:06 PMKevin Kho
Kevin Kho
flow.run()
?Scarlett King
10/06/2021, 7:56 AMflow.run()
works fine. On this flow, I’m using Path from pathlib to handle file path and such. I would have a task that take filepath as an input and would pass filepath= ROOT_DIR / ‘filename.csv’
where ROOT_DIR = Path(__file__).resolve().parent
. So I removed these and set the file path as 1 string, I believe this only occurs when I pass the value to a task function. When I run this inside a task it works fine with no error.
def save_file(df):
file_path = ROOT_DIR / ‘file_name.csv’
df.to_csv(file_path)
Scarlett King
10/06/2021, 8:03 AMdef save_file(df, file_path):
df.to_csv(file_path)
with Flow(‘abc’, storage=azure_store, run_configs=LocalRun()) as flow:
df = query_db(sql_query)
save = save_file(df, file_name=ROOT_DIR / ‘file_name.csv’)
Scarlett King
10/06/2021, 8:06 AMkubernetes_run = KubernetesRun(image=os.environ.get(“IMAGE_URL”), job_template_path=os.environ.get(“JOB_TEMPLATE_PATH”, ROOT_DIR / “job_template.yaml”))
Scarlett King
10/06/2021, 8:07 AMScarlett King
10/06/2021, 8:08 AMROOT_DIR / “job_template.yaml”
with ”./job_template.yaml”
Scarlett King
10/06/2021, 8:09 AMflow.run()
as it can’t find the job_template.yaml fileScarlett King
10/06/2021, 10:40 AMstr(ROOT_DIR / “job_template.yaml”)
Kevin Kho
Scarlett King
10/06/2021, 3:32 PM