Tony Yun
12/07/2021, 7:49 PM@task(log_stdout=True)
def test():
print(f'current folder: {Path.cwd()}')
raise Exception('test exception')
Task 'test': Exception encountered during task execution!
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 863, in get_task_run_state
logger=self.logger,
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 445, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/Users/tonyyun/github/data_operations/dbt_automation/flow.py", line 296, in test
Exception: test exception
, where /Users/tonyyun/github
shouldn’t be there^Kevin Kho
Tony Yun
12/07/2021, 7:57 PMKevin Kho
Kevin Kho
Tony Yun
12/07/2021, 8:12 PMstorage = Docker(
base_image=PREFECT_IMAGE,
registry_url="<http://4924xxx.dkr.ecr.us-east-1.amazonaws.com/prefect/|4924xxx.dkr.ecr.us-east-1.amazonaws.com/prefect/>",
)
PREFECT_IMAGE = "prefecthq/prefect:0.15.5-python3.7"
Kevin Kho
Kevin Kho
stored_as_script=True
and add the files to the containerAnna Geller
stored_as_script=True
, here is an example: https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_no_build/docker_script_kubernetes_run_custom_ecr_image.py
But note that with stored_as_script=True
, you need to build the image yourself and push it to ECR before you register and run the flow. Here are commands you can use to do that.Tony Yun
12/07/2021, 9:51 PM