whats a right way to submit a spark job from prefect task?

I personally use the

subprocess

python module to submit spark jobs and you should be able to use

asyncio.create_subprocess_shell

to execute multiple jobs simultaneously. Prefect also has the built-in

ShellTask

task

how to handle success / fail of job run?