https://prefect.io logo
#prefect-kubernetes
Title
# prefect-kubernetes
m

Manoj Babu

04/02/2023, 8:32 AM
Hi folks, I've written a prefect integration which helps to deploy and monitor spark jobs on kubernetes via spark-on-k8s-operator. You can just simply use your existing
SparkApplication
or
ScheduledSparkApplication
(will be reduced to SparkApplication at run time) manifests in these flows. Delegating the scheduling and retrial functionalities to prefect, it gives one much more flexibility to add other sorts of tasks sequentially or paralelly. Please let me know how it works! Thanks https://tardunge.github.io/prefect-spark-on-k8s-operator/ https://github.com/tardunge/prefect-spark-on-k8s-operator
1
5
🎉 2
🦜 2
🙌 2
15 Views