Hi guys, I'm very curious about prefect. We are bu...
# prefect-community
r
Hi guys, I'm very curious about prefect. We are build our big data architecture, no we use aws glue with pyspark, but we want to use aws emr, so we looking for a orchestration tool. Prefect can submit spark jobs to emr? Thank you
n
Hi @Rubens S么to! Anything you can do with Python, you can do with Prefect, including interfacing with EMR Spark! 馃槃 So yes, you can certainly use Prefect Tasks to orchestrate sending Spark jobs to EMR!
r
But there is no native connector like airflow does?
n
No we don't have a native connector but that sounds like a great addition to the Task Library!