Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

I'm trying to use AWS batch with Prefect using the aws integration. I am able to submit jobs to AWS batch just fine, but they immediately show as completed by prefect. How do I get it to wait until the batch job is completed and basically treat it like any other task where it waits for it to complete and uses the outputs of the batch job for the next task? (Sorry if I'm missing something simple).

I might have figured it out. Do I need to use the batch task along with a client_waiter?

Err actually it looks like batch doesn't have a waiter. If I want this functionality do I need to build it myself?

Hi <@U07GY32TRK2> did you manage to figure this out ?