Hi, I have
• job A runs at 1am daily
• job B runs at 3am Monday, Tuesday, Wednesday
• Job B depends on data from Job A (A pulls in some files on the disk)
Job A/B could either Flow/Task, but I think Flow is best because I could schedule it.
My question is, how do I see the dependency between Job A and Job B?
I cannot create a parent flow and wrap Job A & Job B together, because Job B doesn't always run when Job A runs. So, I cannot see the dependency like that.
How should I approach this? Thank you!
m
Mitch Zink
11/06/2024, 12:54 AM
Have you tried triggers yet? You can set the flow B to kick off when flow A completes successfully
t
Tri
11/06/2024, 3:07 PM
hi, would it show the dependency that B depends on A?
Also, I would like B to start exactly at 3am, not just when A finishes successfully. I don't think there is a time trigger right?
m
Mitch Zink
11/06/2024, 3:54 PM
If it were me, I would chain them so B always ran after A. You can add a simple if else statement at the beggining of the code. Like if today is X day, run the app. If not, do nothing
Mitch Zink
11/06/2024, 3:54 PM
Hope that helps a bit, I would try to tackle it with a mix of scheduling and python
t
Tri
11/06/2024, 4:04 PM
thank you. Ideally I'd like a separate A deployment and B deployment, each with their own schedule. Then somehow link them together, to show the dependency. But sadly it's not doable
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.