Hi there folks, how you doing?
Github Actions deployment question
I'm trying to improve our data team workflow with Prefect and our flows are using
Github Storage
.
I'm facing a problem when "developing" the flows. Since we're using Github Actions, flows are registered to a
development
project when a Pull Request is opened (and updated). But
Github Storage
will use
main/master
as
ref argument. This works fine for production, but won't do while I'm on a development branch.
The workaround I've found is to use a pattern like this:
• Github Actions Workflow with different steps for Push/Pull Request:
◦
Push to main: generate project folder and register the flow
▪︎ I use
prefect register --project $project --path $path
to do this
◦
Pull Request: register the flow with disabled schedule in
development
project
▪︎ I use the
if __name__ = "__main__"
pattern in order to pass a dynamic
GITHUB_HEAD_REF
to
Github Storage ref argument.
It's
working, but I'm thinking if there is way just use the
prefect cli
for this and avoid writing boilerplate code.
The thread has some code snippets and more details!
