Context • Prefect 1.2.4, Tenant = Deliveroo • it’s our massive static flow, which has historically had >2000 nodes, but as of today has just gone over 2500 The error message is

prefect.exceptions.ClientError: [{'path': ['register_edges'], 'message': 'Edges could not be registered - some edges reference tasks that do not exist within this flow.', 'extensions': {'code': 'INTERNAL_SERVER_ERROR'}}]

however that’s 100% not true - we’ve validated this locally, and all the edges + nodes do exist, which makes me think Prefect Cloud is stopping at 2500 tasks somewhere, and then the edges are throwing an error

Flow ID is: https://cloud.prefect.io/deliveroo/flow/db9bef75-eaf9-4656-9222-b2052b0e1e85 Prefect Cloud Tenant ID:

ecf1e74b-ff7b-43f2-b734-c062084a6c3b

in case you can see anything in the registration logs?

We reverted a few additions back to 2498 tasks and it registers fine, but as soon as we go over that, we get the above message, hence the 2500 internal cloud limit hunch

If you could put a breakpoint at https://github.com/PrefectHQ/prefect/blob/1.x/src/prefect/client/client.py#L913 and confirm that the

len(serialized_tasks)

is the number you expect and that

stop

at L925 after the loop completes is greater than that number we should be able to eliminate a client-side issue

Untitled.py

TL;DR - it does appear to be processing the correct number of tasks (2506) across 6x batches, but then when it gets to the first batch of edges we get the ‘task doesn’t exist’ error

I also added in a step in the above register method to write the

serialized_tasks

and

serialized_edges

out to json files, and have since read them into the console and ran a simple comparison to ensure all the edge slugs in the

serialized_edges

exist within

serialized_tasks

, and they do, so I’m certain there’s something going on with the graphQL not actually registering these tasks or something 🤔 Can you see anything from the cloud logs on your side?