https://prefect.io logo
Powered by
Title
j

Jake

05/10/2022, 6:05 PM
Hello! Is flow registration to prefect cloud asynchronous? I am trying to register multiple flows concurrently. Are there any examples of that?
k

Kevin Kho

05/10/2022, 6:12 PM
The API only takes in one Flow so not really. Is it too slow for you? You are the one who has a CI/CD pipeline so you made a way to determine only the Flows with changes right?
j

Jake

05/10/2022, 6:16 PM
yeah that’s me. It’s still too slow 😞. I’m also curious what exactly goes into building the hash generated from 
flow.serialized_hash()
. I’ve noticed that sometimes the hash changes even though there were no changes in the code. Does it have anything to do with the pickle? I noticed that even when there are no code changes, the pickles contents changed.
k

Kevin Kho

05/10/2022, 6:21 PM
That must mean you have something like 
datetime.datetime.now()
in your schedule or something? Some timestamps that changes value from run to run
j

Jake

05/10/2022, 6:24 PM
That could be it actually; let me double check
This works by the way — it’s somewhat faster. 
# where register_flows is a function that calls flow.register()

threads = []
for project in projects:
           t = threading.Thread(
               target=register_flows, args=(project)
           )
           threads.append(t)
           t.start()

for thread in threads:
       thread.join()
k

Kevin Kho

05/13/2022, 7:07 PM
Ah ok you could wrap the flow.register call in async too maybe
j

Jake

05/13/2022, 8:52 PM
Yeah that works! But I’m not sure how to await them from a CLI
k

Kevin Kho

05/13/2022, 9:45 PM
Ah I see. That’s true
#prefect-communityJoin Slack
Powered by