Good morning. I have one question regarding storag...
# prefect-kubernetesn
nicholasnet
05/31/2023, 3:01 PMGood morning. I have one question regarding storage block.
I do not want to store code in s3 bucket but rather store only result for re-tries and such. Do you know what is the best way to achieve that.
Currently I am using Kubernetes Job with custom image which have all the code and dependency. Furthermore, can this be achieved without storage block.
z
Zanie
05/31/2023, 5:03 PM
Zanie
05/31/2023, 5:04 PM
This requires a storage block and does not store code.
n
nicholasnet
05/31/2023, 6:07 PMThank you, Can we use Local storage for Kubernetes Job? I do not think so but I would like to verify it.
z
Zanie
06/01/2023, 1:44 PM
Yes you can. If you want it to persist longer, you’d need to set up a persistent volume mount though.
n
nicholasnet
06/01/2023, 2:43 PMIs there any example somewhere using such persistence inside flow.
z
Zanie
06/01/2023, 2:44 PM
Like how to create a persistent volume or… ?
n
nicholasnet
06/01/2023, 2:45 PMNo how to use it in
flowz
Zanie
06/01/2023, 2:45 PM
The documentation I linked covers that
n
nicholasnet
06/01/2023, 2:45 PMoh ok sorry I will take a look at it.
nicholasnet
06/01/2023, 2:46 PMSo, basically its just LocalFileSystem then right?
z
Zanie
06/01/2023, 2:50 PM
Yep!
Zanie
06/01/2023, 2:51 PM
That’s even what we use by default, so you could just change the default storage setting
Zanie
06/01/2023, 2:51 PM
PREFECT_LOCAL_STORAGE_PATHn
nicholasnet
06/01/2023, 2:51 PMHow do I change that?
nicholasnet
06/01/2023, 2:51 PMIs it a environment variable?
z
Zanie
06/01/2023, 2:52 PM
Yes that is a way to change it
Zanie
06/01/2023, 2:52 PM
You can also use
prefect config setn
nicholasnet
06/01/2023, 2:52 PMok awesome I will give it a try.
nicholasnet
06/01/2023, 2:52 PMThanks
nicholasnet
06/01/2023, 5:56 PMDo I need to add PV to agent as well?
z
Zanie
06/01/2023, 6:02 PM
Nope
👍 1
2 Views