https://prefect.io logo
#prefect-community
Title
# prefect-community
c

Chris Keeley

04/11/2022, 1:09 PM
Hi, we have a requirement for triggering a flow in one tenant A after another flow running in tenant B completes. this doesn't seem to be how prefect is designed to work (?), but other requirements have made this tenant architecture necessary. is there any official support for this pattern, or will I have to write my own connector? I have looked at taking from the
wait_for_flow_run
method for creating flow of flows, but this seems designed to work with one tenant
k

Kevin Kho

04/11/2022, 1:58 PM
This is hard because you will need 2 Clients to juggle those tenant connections, and then you can do something like:
Copy code
client = Client(api_key=other_tenant_key)
client.create_flow_run(flow_run_id)
but all of the tasks are designed with one tenant in mind because they just pull the Client from context. For example,
wait_for_flow_run
uses a Client that it just pulls. This means that if you wanted to do something like
wait_for_flow_run
, you’d need to write a new task that can use another Client. So triggering a flow seems easy but the waiting seems hard. If you just have to trigger, it feels doable.
👍 1
a

Anders Segerberg

04/11/2022, 3:07 PM
On this note -- I had to fork and modify
wait_for_flow_run
because it had a default timeout of 12 hours, and I have longer runtimes. Will this limitation ever be made configurable?
k

Kevin Kho

04/11/2022, 3:14 PM
I think it can be made configurable. Trying to find the code path. Where does it say 12 hours? code here
Ah gotcha. I think we can open an issue for it
a

Anders Segerberg

04/11/2022, 4:13 PM
Since I already modified the relevant code, should I just make a PR and link it to this issue?
k

Kevin Kho

04/11/2022, 4:14 PM
Yeah totally
a

Anders Segerberg

04/11/2022, 4:15 PM
ok. do i need to "claim" the issue to indicate it's being worked on?
k

Kevin Kho

04/11/2022, 4:17 PM
No need really, but you can if you want. Can you though? I think just comment and say you’ll make a PR for it on the issue
a

Anders Segerberg

04/11/2022, 4:18 PM
cool
c

Chris Keeley

04/13/2022, 10:37 AM
thanks @Kevin Kho, for pointing me in the right direction. I got a proof of concept that waits on a task in another tenant working. understanding that the methods are getting the tenant from disk because of
client = Client()
was helpful. I just had to write a task which polls using a provided client, giving a RETRY signal if it's not ready. and extracted some graphql queries from the source to do it against my own client.
it's a bit hamfisted, it would be nicer if client was a default argument rather than hardcoded, as it makes some of this stuff quite stateful
8 Views