John Grubb
11/30/2020, 7:25 PMUnexpected error while running flow: KeyError('Task slug foo-3 not found in the current Flow; this is usually caused by changing the Flow without reregistering it with the Prefect API.')
It's always the same task for both pipelines - I've split out a common task to load data to BigQuery into another file and this is the task that fails. It will run fine for a few days and then randomly decide that it can't find this task, even though the exact same task is called two other times during each pipeline, foo-1 and foo-2. I'm wondering if I'm the first person to have their install randomly lose tasks..Kyle Moon-Wright
11/30/2020, 7:53 PMJohn Grubb
11/30/2020, 7:58 PMKyle Moon-Wright
11/30/2020, 8:00 PMJohn Grubb
11/30/2020, 8:01 PMKyle Moon-Wright
11/30/2020, 8:03 PMJohn Grubb
11/30/2020, 10:02 PMAmanda Wee
11/30/2020, 10:05 PMI suppose I need to better understand how to have agents only listen for flows that are intended for their particular environment.Ah, I believe that is done through labels on the agents and the RunConfig objects (previously the environments). Can refer to: https://docs.prefect.io/orchestration/flow_config/run_configs.html#labels
John Grubb
11/30/2020, 10:10 PMBrad
01/06/2021, 10:24 PMTask slug foo-3 not found in the current Flow
?
My setup:
• Prefect server running on kubernetes via the helm chart (agent included)
• Docker storage (I have a base image containing my code, and building a “flows” tag via docker-storage from that)
• KubernetesRun run config
• Deployed via gitlab CI
I’m pushing changes to code/flow and using the new flow.register(idempotency_key=flow.serialized_hash()) which kicks off the following CI process:
• Build a new base image for the repo, run tests
• Run the register step, building the docker storage (with a “flows” tag) with base image from previous step
This is reasonably complicated setup, but I can’t see any reason why it would be throwing the Task slug foo not found in the current Flow
(totally fresh build every time). Open to any ideas on how to debugAmanda Wee
01/06/2021, 10:33 PMBrad
01/06/2021, 10:34 PM