Curious if anyone has used prefect with real-time data pipeline curious what's your setup like? Also, has anyone used prefect to run Spark jobs?

This might be a good place to read on how companies use Prefect in production https://www.prefect.io/guide/case-studies/ for spark, there are some users who e.g. trigger a Spark job using https://github.com/PrefectHQ/prefect-databricks