Manoj Babu
04/02/2023, 8:32 AMSparkApplication
or ScheduledSparkApplication
(will be reduced to SparkApplication at run time) manifests in these flows. Delegating the scheduling and retrial functionalities to prefect, it gives one much more flexibility to add other sorts of tasks sequentially or paralelly. Please let me know how it works!
Thanks
https://tardunge.github.io/prefect-spark-on-k8s-operator/
https://github.com/tardunge/prefect-spark-on-k8s-operator