https://prefect.io logo
Title
a

Atul Anand

04/04/2022, 7:09 PM
Task was destroyed but it is pending! task: <Task pending name='Task-36427' ! Sometime I got this error. Note: I have millions of task running parallely.
k

Kevin Kho

04/04/2022, 7:10 PM
Millions!? Could you show me a screenshot or some logs?
a

Atul Anand

04/04/2022, 7:13 PM
k

Kevin Kho

04/04/2022, 7:17 PM
This is Prefect 1 on Dask right? That specific log is not a Prefect log though
What is your task code doing?
a

Atul Anand

04/04/2022, 7:20 PM
I just have list of 100000 objects which contains different attributes. I am using map function on this list for analysing sentiment of each and every object seperately
@Kevin Kho Absolutely! I am using Prefect 1.
k

Kevin Kho

04/04/2022, 7:21 PM
I think this is more of a Dask error
Like this
a

Atul Anand

04/04/2022, 7:23 PM
Hahahaha! Gone through this😃 I thought some have/maybe encountered this in our community.
k

Kevin Kho

04/04/2022, 7:23 PM
No first time seeing it here. I don’t know if you need to try to slow down submission a bit
a

Atul Anand

04/04/2022, 7:30 PM
Slowing Down Makes it successful. Just want to know why this is happening. We are working on a product in which at a given point of time we need to process millions of tasks.
k

Kevin Kho

04/04/2022, 7:35 PM
I wouldn’t know for this scale. Maybe you can asking the Dask Discourse? But it’s not Dask specific, i think it’s asyncio specific. Still though, they might be a good place to ask, or maybe the asyncio github issues?
a

Atul Anand

04/04/2022, 7:38 PM
@Kevin Kho, Hey Kevin ! Thanks for the help.