https://prefect.io
Join Slack
Hi Everyone, Had the same issue with k8 job deployments that was first found here `Failed to submit...
# ask-community
b
Hi Everyone, Had the same issue with k8 job deployments that was first found here 
Failed to submit flow run to infrastructure
https://prefect-community.slack.com/archives/CL09KU1K7/p1663166587656539 Details in thread
kubernetes 1
✅ 2
k8 agent log
Copy code
21:24:34.664 | INFO    | prefect.agent - Submitting flow run 
'3c1ba6ba-ed19-45f2-9d4a-144dd57aad16'
21:24:35.179 | ERROR   | prefect.agent - Failed to submit flow run 
'3c1ba6ba-ed19-45f2-9d4a-144dd57aad16' to infrastructure.
21:24:35.181 | INFO    | prefect.agent - Completed submission of flow run 
'3c1ba6ba-ed19-45f2-9d4a-144dd57aad16'
our issue was due to the length of our flow run names 2 flow run names 
vAT_T_TX_TX_Broadband_Availibility_June_2022_Map_Data_RawFile2_CSV_sec6
AT_T_TX_TX_Broadband_Availibility_June_2022_Map_Data_RawFile2_CSV_6
are more than 63 characters and get cut off to 
vat_t_tx_tx_broadband_availibility_june_2022_map_data_rawfile2
when creating k8 job names and k8 job labels
this causes k8 to complain and fail to create jobs (because pods with similar names already exist as far as I know) we've changed the flow run names and our issue was resolved. this took a while to debug, so I wanted to post here in case someone had a similar issue
🙌 1
🙌 2
z
The fix for this will be released on Thursday https://github.com/PrefectHQ/prefect/pull/7528
Although yours may be slightly different since the name isn’t ending in nonalphanumeric characters
b
yeah I think this is a different issue, but good thing to post 7528 here
z
It’s because the names already exist? 😕 This will be quite hard to solve if the K8s API is just throwing 500s
b
I'm not exactly sure if there is an easy fix for this, but thank you for looking anyway @Zanie! I think this an edge case. As long as my teams flow run names are less than 63 and they are all unique, I don't think we'll see this issue again
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by