Ofir
04/28/2023, 5:13 PMimport os
PERSISTENT_OUT_DIR = '/my_data'
out_path = os.path.join(PERSISTENT_OUT_DIR, 'my_output.csv')
with open(out_path, 'w') as f:
f.write('my,cool,csv,header')
And then kubectl cp
or e.g. use a filebrowser to access them via a Web UI.
However, I am sure that there are standard ways to upload generated files during the workflow runs from Prefect
and I’d love to learn about the idiomatic approaches.
I am guessing Prefect has support through its SDK to access S3 / Azure Blob Storage / Azure Files, etc.
Your thoughts?Henning Holgersen
04/28/2023, 5:29 PMOfir
04/28/2023, 5:30 PMHenning Holgersen
04/28/2023, 5:42 PMOfir
04/28/2023, 5:47 PMHenning Holgersen
04/28/2023, 5:54 PMOfir
04/28/2023, 5:57 PMHenning Holgersen
04/28/2023, 5:58 PMOfir
04/28/2023, 5:58 PMHenning Holgersen
04/28/2023, 5:59 PMOfir
04/28/2023, 6:03 PMHenning Holgersen
04/28/2023, 6:25 PM