Channels
pacc-may-31-2023
prefect-ai
pacc-clearcover-june-12-2023
marvin-in-the-wild
data-ecosystem
geo-israel
geo-japan
prefect-cloud
ppcc-may-16-2023
prefect-azure
prefect-docker
prefect-recipes
gratitude
geo-nyc
geo-bay-area
geo-boston
geo-london
geo-dc
geo-chicago
geo-berlin
geo-texas
geo-seattle
geo-colorado
prefect-community
data-tricks-and-tips
prefect-aws
prefect-gcp
introductions
find-a-prefect-job
prefect-dbt
random
events
ask-marvin
show-us-what-you-got
prefect-getting-started
prefect-integrations
prefect-contributors
best-practices-coordination-plane
announcements
prefect-server
prefect-ui
prefect-kubernetes
Title
t
Tomás Emilio Silva Ebensperger
06/05/2021, 2:39 PMHello one question. I set up a 1800 second timeout for a task, but it got stuck and didnt finish after that time out, it kept on waiting for hours and hours.
@task(log_stdout=True, state_handlers=[handler], timeout=1800)k
Kevin Kho
06/05/2021, 3:40 PMHi @Tomás Emilio Silva Ebensperger, what is that task doing? This seems like something in the task was hanging
t
Tomás Emilio Silva Ebensperger
06/05/2021, 4:09 PMi am hitting an api that sometimes takes quite a bit to het the 200 http status. i want to fail the flow if it takes more than the specified time.
k
Kevin Kho
06/05/2021, 4:12 PMThis seems like it should work. I’d have to check with the team and get back to you on Monday.
t
Tomás Emilio Silva Ebensperger
06/05/2021, 4:15 PMawesome man,, im goingt o try with shorter timeouts and test a little, thanks man!!
k
Kevin Kho
06/08/2021, 12:51 AMSorry I forgot to ask the team today. Will leave a message with them and get back to you tomorrow.
t
Tomás Emilio Silva Ebensperger
06/08/2021, 1:04 AMthank you!
k
Kevin Kho
06/08/2021, 1:04 AMDid you try the shorter timeouts and see the same behavior?
t
Tomás Emilio Silva Ebensperger
06/08/2021, 1:05 AMi did
and it worked
so it is very weird
i did a short 10 second timeout, and it stopped like it should
z
Zanie
06/08/2021, 3:18 PMHey @Tomás Emilio Silva Ebensperger -- could you try turning on DEBUG level logs with the long timeout? Timeouts are a bit complicated and the logs will help us figure out where it's hanging. Could you also provide the output of
prefect diagnosticst
Tomás Emilio Silva Ebensperger
06/09/2021, 4:28 PM{
"config_overrides": {
"cloud": {
"agent": {
"auth_token": true
}
},
"context": {
"secrets": false
}
},
"env_vars": [],
"system_information": {
"platform": "Linux-5.4.0-51-generic-x86_64-with-glibc2.29",
"prefect_backend": "cloud",
"prefect_version": "0.14.12",
"python_version": "3.8.5"
}
}ok i set up the debug level, once i get the error , i will send you the logs
thank you
z
Zanie
06/09/2021, 5:12 PMThere's also a fix in 0.14.16 for some edge-case timeout issues https://github.com/PrefectHQ/prefect/pull/4384
t
Tomás Emilio Silva Ebensperger
06/09/2021, 5:44 PMok 30 minutes have passed and no luck with the timeout
it was running that checking flow run state every 4 seconds and then it stopped
( i am not multithreading and the ram and cpy usage seem normal)
z
Zanie
06/09/2021, 6:41 PMHey @Tomás Emilio Silva Ebensperger -- could you share the logs when the task is set up? There should be some things about timeout handling
t
Tomás Emilio Silva Ebensperger
06/09/2021, 7:33 PM