Channels
pacc-may-31-2023
prefect-ai
pacc-clearcover-june-12-2023
marvin-in-the-wild
data-ecosystem
geo-israel
pacc-june-14-2023
geo-japan
prefect-cloud
ppcc-may-16-2023
prefect-azure
prefect-docker
prefect-recipes
gratitude
geo-nyc
geo-bay-area
geo-boston
geo-london
geo-dc
geo-chicago
geo-berlin
geo-texas
geo-seattle
geo-colorado
prefect-community
data-tricks-and-tips
prefect-aws
prefect-gcp
introductions
find-a-prefect-job
prefect-dbt
random
events
ask-marvin
show-us-what-you-got
prefect-getting-started
prefect-integrations
prefect-contributors
best-practices-coordination-plane
announcements
prefect-server
prefect-ui
prefect-kubernetes
Title
t
Tadej Svetina
01/08/2023, 9:28 PMHi, I have an issue: I have a task, where basically constitutes of polling an API every 30 seconds. I use
asyncio.sleep👀 1
Ok, I have a nice replication example. This occurs only when using
submitimport asyncio
import prefect
@prefect.task
async def sleepy():
while True:
print("Hi")
await asyncio.sleep(10)
@prefect.flow
async def sleepy_flow():
await sleepy.submit()
asyncio.run(sleepy_flow())Without using submit, I am getting normal (near 0) CPU usage
g
George Coyne
01/10/2023, 3:50 PMIt looks like this is expected behavior
t
Tadej Svetina
01/10/2023, 4:13 PMThis is not the same - the issue you posted uses asyncio.sleep(0), so there would be constant polling. I use asyncio.sleep(10), which means that the event loop should simply ignore the task for 10 seconds
z
Zanie
01/10/2023, 4:13 PM👍 yeah this is not proper behavior — I’m changing how we handle concurrency so hopefully it is addressed by that.
It seems possible that we’re encountering one of the spin locks at https://github.com/PrefectHQ/prefect/blob/main/src/prefect/task_runners.py#L317-L327