Madison Schott
07/20/2022, 3:34 PMKevin Kho
07/20/2022, 4:15 PMMadison Schott
07/20/2022, 4:18 PMKevin Kho
07/20/2022, 4:18 PMMadison Schott
07/20/2022, 4:20 PMKevin Kho
07/20/2022, 4:23 PMMadison Schott
07/20/2022, 5:03 PMKevin Kho
07/20/2022, 5:05 PMMadison Schott
07/20/2022, 5:14 PMKevin Kho
07/20/2022, 5:17 PMMadison Schott
07/21/2022, 1:59 PMIn our case, we make a simple tweak at the end of our script to tell this flow to use the run_spark_job task as its reference task:
flow.set_reference_tasks([submitted])
[submitted]
in the docs supposed to be the run_spark_job
?Kevin Kho
07/21/2022, 8:49 PMMadison Schott
07/21/2022, 8:51 PMKevin Kho
07/21/2022, 8:53 PM