Is there a way in Prefect to set Deployment B to be dependent on Deployment A? (cross DAG deps) I h...

Ofir

08/15/2023, 7:07 PM

Is there a way in Prefect to set Deployment B to be dependent on Deployment A? (cross DAG deps) I have a daily routine ETL task which is Prefect Deployment A, runs every day at 02:00 am. Then I have one or more Deployment B’s that should run on a daily basis, if-and-only-if Prefect Deployment A had a successful flow run. Is it possible to set dependencies between Deployment B and Deployment A? If so, how? Thanks!

✅ 1

Geoffrey Keating

08/15/2023, 7:13 PM

I think this would be a good application for automations or `run_deployment`. The former would be configured in Prefect Cloud and the latter being in code

Ofir

08/15/2023, 7:16 PM

Thanks @Geoffrey Keating! I am using the self-hosted version of Prefect, and not Prefect cloud.

Ofir

08/15/2023, 7:16 PM

Is there a way to set cross DAG dependencies? i.e. tell the Prefect server through REST API that Deployment B may start only if there was a previous successful flow run of Deployment A? or something along these lines?

Ofir

08/15/2023, 7:17 PM

Perhaps setting a trigger for Deployment B or an event-driven runs

Geoffrey Keating

08/15/2023, 7:18 PM

Not sure about that, anyone else got an answer?

🙌 1

Serina

08/15/2023, 11:35 PM

I think what @Jake Kaplan mentioned yesterday is applicable here, by using an

on_completion

hook 🙂: https://prefect-community.slack.com/archives/CL09KU1K7/p1692046985857219?thread_ts=1692046555.221379&cid=CL09KU1K7

➕ 2

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.