Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi everyone! Just want to ask whether or not prefect can have an easy integration with AWS EMR Serverless and Pyspark. Thanks!!

hi <@U09R5BM808H> - those integrations do not exist today, but you could make them if you wanted to!

if you ask claude to `git clone <https://github.com/prefecthq/prefect>` and look at `src/integrations/*` and make a new library `prefect-{name}` , it should do a pretty good job

hmm do you think it shuold be under the prefect aws libray since it's aws emr servereless?

yes! if you want to make a contribution then that's probably where it should go

but if you can't wait for that to happen/get reviewed/merged/released then you could fork/make your own integration

sounds good thanks!! will start reading the code base more before i start making the contribution