Channels
#prefect-community
Title
w
Wei Mei
03/18/2022, 6:05 PMHi, I am making my first pass at using Github actions to CI. I have reached something I don’t fully understand. On pull_request of a feature/** branch, I run actions that:
1. checkout the branch
2. create a project in prefect cloud
3. register the flow.py with a --label dev
when I run the flow manually, i get an error:
Unexpected error while running flow: KeyError('Task slug connect_source-1 is not found in the current Flow. This is usually caused by a mismatch between the flow version stored in the Prefect backend and the flow that was loaded from storage.\n- Did you change the flow without re-registering it?\n- Did you register the flow without updating it in your storage location (if applicable)?')k
Kevin Kho
03/18/2022, 6:06 PM
What is your storage? This error normally comes if your registered flow does not match the Github repo code for example
w
Wei Mei
03/18/2022, 6:07 PMHi, I am using the gitHub storage.
If i merge to main, and then try to run the flow, it works as expected.
k
Kevin Kho
03/18/2022, 6:09 PM
When you register, is the storage using the main branch or feature branch?
w
Wei Mei
03/18/2022, 6:13 PM Copy code
with Flow(
"hello-flow-example",
storage = GitHub(
repo="wei/prefect-extract",
path="flows/flow.py",
access_token_secret="GITHUB_ACCESS_TOKEN",
),this is what is in my storage definition
k
Kevin Kho
03/18/2022, 6:14 PM
So the issue is that it’s using the main branch for the run but it’s registered with the feature branch code right?
w
Wei Mei
03/18/2022, 6:14 PMyes. it sounds like that is indeed the case.
Is that something I need to change in my flow?
or my github action?
sorry, this is my very 1st attempt.
k
Kevin Kho
03/18/2022, 6:22 PM
No worries! I’m not 100% sure either but I think you have to merge then register for this to work. I think you can either
1. not register and run the flow to test. it could just be unit tests on the tasks
2. register and test separately outside of the CI/CD? in this registration and testing, you could use the feature branch
I think the problem is clear. It’s not something wrong with your flow or action. There is something wrong with the process. Does that make sense?
w
Wei Mei
03/18/2022, 6:25 PMok cool.
i probably trying to hard to make a super-CI.
k
Kevin Kho
03/18/2022, 6:25 PM
A bit trickier to set up but you can register again with the “main” storage after testing the feature branch is a test env with registering and running. Just a bunch of ideas
You have the same problem as Aram in the post above, maybe you can try asking him what Storage he uses and how he does it haha
w
Wei Mei
03/18/2022, 6:26 PMSure.
thanks for your time. I read this one too: https://github.com/PrefectHQ/prefect/discussions/4042
but they little advanced for me at moment 🙂.
👍 1
m
Matthias
03/18/2022, 6:58 PMI think it just comes down to what you want to do/test when a PR is created. At that point, the main focus should be automated testing to ensure the code is working as expected. So the question is: do you need to register the flow for testing?
👍 1
w
Wei Mei
03/18/2022, 7:06 PMI believe as Kevin mentioned looking for something similar to Aram. At this point, I would accept the successful regsiter with no error and the ability to click run on the dashboard and get a good run from this project before i merge it to main.
👍 1
m
Matthias
03/18/2022, 7:40 PMI think registering the flow under a new project (in case you don't have a multi-tenant setup) is the easiest thing you can do in this case (just make sure you register with the storage pointing to the feature branch instead of the main branch). Would that work?
w
Wei Mei
03/18/2022, 7:42 PMthat is the part I am not aware of how to accomplish.
Is
refm
Matthias
03/18/2022, 11:04 PM@Wei Mei that is correct, see https://docs.prefect.io/api/latest/storage.html#github.
✅ 1
17 Views