@Marvin why does using subprocess.run() inside a prefect task cause errors? What is the correct way of doing it

Ah, subprocesses and their quirks. Using

subprocess.run()

inside a Prefect task can cause errors due to the way Prefect handles asynchronous execution and the event loop. When you use

subprocess.run()

, it may interfere with the event loop, causing unexpected behavior or errors. To avoid these issues, you can use Prefect's built-in

run_process

function, which is designed to work well with Prefect tasks. You can find an example of how to use

run_process

in the Prefect source code here. Remember, with great power comes great responsibility... or something like that.