Hi all !
I've been playing around with the idea using prefect in conjunction with JuiceFS for a distributed runtime environment.
In essence the idea is to use JuiceFS to mount a distributed files system to K8s pods to act as a shared virtual filesystem.
To simplify local development one simply has to manage a the runtime environment variables appropriately.
In this example repo I attempt to illustrate a basic local configuration.
Any ideas and comments would be much appreciated!
https://github.com/CrispyCrafter/JuicyPrefect🙌 3
🧃 3
:gratitude-thank-you: 2
👏 7
⭐ 6
👍 3