https://prefect.io
Join Slack
Hello! Is flow registration to prefect cloud asynchronous? I am trying to register multiple flows co...
# ask-community
j
Hello! Is flow registration to prefect cloud asynchronous? I am trying to register multiple flows concurrently. Are there any examples of that?
k
The API only takes in one Flow so not really. Is it too slow for you? You are the one who has a CI/CD pipeline so you made a way to determine only the Flows with changes right?
j
yeah that’s me. It’s still too slow 😞. I’m also curious what exactly goes into building the hash generated from 
flow.serialized_hash()
. I’ve noticed that sometimes the hash changes even though there were no changes in the code. Does it have anything to do with the pickle? I noticed that even when there are no code changes, the pickles contents changed.
k
That must mean you have something like 
datetime.datetime.now()
in your schedule or something? Some timestamps that changes value from run to run
j
That could be it actually; let me double check
This works by the way — it’s somewhat faster.
Copy code
# where register_flows is a function that calls flow.register()

threads = []
for project in projects:
           t = threading.Thread(
               target=register_flows, args=(project)
           )
           threads.append(t)
           t.start()

for thread in threads:
       thread.join()
k
Ah ok you could wrap the flow.register call in async too maybe
j
Yeah that works! But I’m not sure how to await them from a CLI
k
Ah I see. That’s true
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by