Prasanth Kothuri
04/19/2022, 3:51 PMTask 'copy_from_s3_to_sftp': Exception encountered during task execution!
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/prefect/engine/task_runner.py", line 876, in get_task_run_state
value = prefect.utilities.executors.run_task_with_timeout(
File "/usr/local/lib/python3.9/dist-packages/prefect/utilities/executors.py", line 454, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "flows/k8s/my_flow_name.py", line 46, in copy_from_s3_to_sftp
SystemError: unknown opcode
Kevin Kho
04/19/2022, 3:52 PMPrasanth Kothuri
04/19/2022, 3:54 PMKevin Kho
04/19/2022, 3:58 PMPrasanth Kothuri
04/19/2022, 3:59 PMKevin Kho
04/19/2022, 4:46 PMwith Flow(...) as flow:
a = create_flow_run(..)
b = wait_for_flow_run(a, raise_final_state=True)
c = create_flow_run(..., upstream_tasks=[b])
raise_final_state being the important part so failure gets raised and then propagatesPrasanth Kothuri
04/19/2022, 5:59 PMwith Flow("priority_report", storage=storage, schedule=schedule) as f:
flow_a = create_flow_run()
wait_for_flow_a = wait_for_flow_run(flow_a, raise_final_state=True)
copy_from_s3_to_sftp(upstream_tasks=[wait_for_flow_a])
Kevin Kho
04/19/2022, 6:04 PMtask_one
, task_two
, task_three
but you only want task_two
?
Or do you mean a parent Flow that calls one sub flow and one task? Because what you have looks good to mePrasanth Kothuri
04/19/2022, 6:06 PMparent Flow that calls one sub flow and one task
, strangely I am getting this error
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/prefect/agent/agent.py", line 391, in _deploy_flow_run
deployment_info = self.deploy_flow(flow_run)
File "/usr/local/lib/python3.8/dist-packages/prefect/agent/local/agent.py", line 133, in deploy_flow
env = self.populate_env_vars(flow_run, run_config=run_config)
File "/usr/local/lib/python3.8/dist-packages/prefect/agent/local/agent.py", line 190, in populate_env_vars
else os.getcwd()
FileNotFoundError: [Errno 2] No such file or directory
Kevin Kho
04/19/2022, 6:08 PMPrasanth Kothuri
04/19/2022, 7:09 PMKevin Kho
04/19/2022, 7:12 PMPrasanth Kothuri
04/19/2022, 8:16 PM