juandavidlozano05/15/2023, 7:24 PM
Taylor Curran05/15/2023, 7:37 PM
Ross Leung05/18/2023, 8:45 PM
the task calls a spark job. Both tasks started concurrently according to the Prefect Server UI, but the 2nd failed with a long error message, and one of the line in the error log says “SparkContext was shut down”. Any idea why? @Taylor Curran any tip would be appreciated!
@flow(task_runner=ConcurrentRunner()): task.submit(param=1) task.submit(param=2)
Taylor Curran05/22/2023, 2:37 PM