Hi Prefect community ! :wave: Could someone (or mo...
# prefect-community
n
Hi Prefect community ! 👋 Could someone (or more) please share any best practices in regards to registering a flow? If I have my code in a repository and then this repository deployed to a server where prefect is running (server and agent) what is an elegant way of registering a flow? currently i need to open some kind of python (ipython or notebook) and register manually each flow (and flow version). Surely there must be a more elegant way?: )
r
I second that. We are also currently registering flows manually, but I would like to set up a deploy stage using GitLab CICD to register flows. I am also looking forward to reading from others who have already successfully done this 🙂
👍 2
n
Hi @Robin If you were to set up a deploy stage, how would that work?
r
If it works out as I would like it, the flow would be reregistered whenever pushing to a (specific) branch on GitLab.
Does that answer your question or are you interested in the technical details?:
n
No, I don’t quite understand technically how I would approach it.
for example: do you wrap the
flow
generation into a function, which you call somehow? What is your flow version has not changed (do you still register); how is the actual registration done, etc.
r
Ah, OK! I was just thinking of running the
flow.py
file on GitLab using python whenever all previous stages have succeeded (i.e. all tests successful).
n
so your
flow.py
contain
flow.register()
?
👍 1
I personally think this may help deployment: https://github.com/PrefectHQ/prefect/issues/3433
r
Hmm, it's true that gitlab would reregister all the flows whenever a new commit is pushed to the respective branch (and all tests succeed), which might be unwanted... Question being why this would be bad?
n
It’s not gitlab that does it; it’s
flow.register()
which does it (just to clarify in case of confusion). I think there is no need to have different versions which are consecutive and identical. It’s just clutter. Separately (and neatly) it does help with deployment because in case anyone deploys by running
flow.register()
(say a cronjob or CI or whatever) then it will only register different flows.
r
Yeah, that's understood. I agree that unnecessary clutter should be avoided and that the functionality that you suggest makes sense.
👍 1
n
If that’s the case, feel free to support on the Issues (no pressure : D )
r
😄 Has not such a high priority for us atm. Good luck with it!
👍 1
j
We are working on a blog post on this particular topic. We make us of gitpython to detect changes in flow files, load the flow objects from each file, and then run
flow.register()
Each flow is stored in a separate folder with a configuration YAML that lets people define a bunch of options, like the flow project name, additional Python dependencies, etc…..
😍 1
👍 2
n
Pleaae share here when ready. Would love to read it.
j
@Jacob Blanco Is that blog post available?
j
Not yet, other priorities came up. Let me see if we have something rough that I can share with you.
🙏 1