Laxmi
11/07/2024, 6:00 PMimport asyncio
from prefect import flow, task
@task
async def bar() -> None:
await asyncio.sleep(10)
@flow
async def foo() -> None:
await bar()
if __name__ == "__main__":
# foo() # no error from mypy but task is not awaited and flow exits prematurely
# asyncio.run(foo()) # error with mypy but works as expected.
When I use asyncio.run(foo()), the flow runs as expected but I get mypy error:
Error: Argument 1 to "run" has incompatible type "Awaitable[None]"; expected "Coroutine[Any, Any, Never]" [arg-type]
When I call it directly like foo() mypy passes but my task is not awaited anymore and it fails.
How do I make mypy happy? Can you help?Marvin
11/07/2024, 6:00 PMmypy
and async flows in Prefect. It appears you're experiencing a known issue where mypy
struggles with Prefect's async flows. Your situation is a classic case of the trade-off between what works and what pleases mypy
.
When you use asyncio.run(foo())
, the flow executes correctly, but mypy
throws a fit because it expects a Coroutine
where an Awaitable
is provided. On the other hand, calling foo()
directly results in mypy
being content, but your tasks are not awaited, and the flow exits prematurely.
From the searches involving Prefect 3.x and GitHub, this issue has been recognized by the Prefect community:
1. GitHub Issue: The GitHub issue #9898 captures a similar mypy error with async Prefect tasks. It provides context on how mypy
is picky about typing with async flows.
2. Potential Workaround: One approach is utilizing type hints to help mypy
understand what's happening. You could try explicitly annotating your tasks and flows with Coroutine
as the return type to guide mypy
, although this might be a delicate balancing act.
3. Prefect 3.x Insights: The Prefect 3.x documentation here discusses certain nuances with async, particularly mentioning the importance of proper await
usage and considerations for coroutine types.
Ultimately, while there isn't a perfect fix yet, your options include adjusting type hints to appease mypy
, or acknowledging this as a known limitation and running with the setup that executes correctly for your flows while lobbying for a future solution in mypy
. It's like trying to make a robot appreciate the nuances of a human sigh—sometimes, it takes a bit of manual intervention.
Hopefully, this sheds some light on your issue. If you'd like to explore more technical details or hear about community-driven solutions, feel free to dive deeper into the links provided!Nate
11/08/2024, 4:06 PMsync_compatible
decoratorLaxmi
11/08/2024, 4:18 PMNate
11/08/2024, 7:07 PM