Hey all! I'm really struggling to register a very ...
# ask-community
d
Hey all! I'm really struggling to register a very large flow (~1200 tasks) to cloud- I'm hitting an
API_ERROR
on flow.register(). I had this issue intermittently when I last worked on this project (late March), related to this post, but it used to register after 1 or 2 retries. However now I can't register at all - I've tried 6 times and still can't register the flow. I'd be very grateful for any advice please! Prefect version 0.14.14, docker storage pushing to ECR, full error message in thread!
Copy code
prefect.utilities.exceptions.ClientError: [{'path': ['create_flow_from_compressed_string'], 'message': 'Unable to complete operation. An internal API error occurred.', 'extensions': {'code': 'API_ERROR'}}]
Untitled.sh
z
Hey @David Elliott -- we've made some recent changes to this that we thought would squash this issue for good but it appears not! We can see where this is failing in the backend and are investigating the cause / fix.
Do you know how many edges your flow has?
👀 1
d
Hey! Yeah just checked, it has 1225 nodes (tasks) and 2922 edges (dependencies)
Thanks for investigating! 🙏 let me know if I can be of any help! Happy to slack over any further details / teat stuff my end if helpful
z
Impressive flow 🙂 we're working on a patch to adjust the batch size (which may have just been a little ambitious)
🙌 1
I'll let you know when it's live
d
Awesome, thank you!
z
Unfortunately the fix is ready but is being delayed by a disruption in a service we use 😞
d
Assuming circleCI? We have the same problem! Thanks for the update 🙂
z
Yeah, I imagine they're having quite the day
d
Just to update that my flow seems to be registering first time now, so looks like that's fixed it. Thank you! 🎉
z
Our fix was released ~20 hours ago so I'm surprised it worked but definitely let us know if you run into any more issues!
d
(I only tested it 7 hours ago, early Monday morning so that aligns with your test being live 🙂 ) Will do, thanks!
z
Ah I misread the timestamp on your message; glad it's working 🙂