I m wondering if anyone has thoughts on writing a flow with Prefect Community #ask-community

I'm wondering if anyone has thoughts on writing a ...

Kevin Grismore

07/31/2022, 5:07 PM

I'm wondering if anyone has thoughts on writing a flow with a combination of sync and async tasks. or perhaps a flow of flows, with some sync and some async? I currently have a flow where the first task needs to be async because it's calling coroutines in another async framework, but I'd like the following tasks to not be async - especially because if I define all the tasks as async, even though I can ask them to

wait_for

the other tasks, they are still visualized as happening concurrently, which makes the visual less fun.

Kevin Grismore

07/31/2022, 5:10 PM

If I do something like this:

Copy code

@task
async def do_something_async():
    #do async stuff

@task
def do_something_sync():
    #do sync stuff

@flow
async def sync_async_mix():
    await do_something_async()
    do_something_sync(wait_for=[do_something_sync])

my flow will crash, so this definitely isn't the way to go

Kevin Grismore

07/31/2022, 5:10 PM

I am new to async so I'm probably doing something dumb here

Kevin Grismore

07/31/2022, 5:18 PM

Maybe to write the question more succinctly, is it the case that any flow with an async task must have all async tasks, and therefore the visualization of the flow is going to represent all tasks as happening concurrently as children of the flow?

Anna Geller

07/31/2022, 5:59 PM

quick answer: you can mix sync and async tasks in a flow, but afaik you can't mix sync and async flows

🙌 1

Anna Geller

07/31/2022, 7:30 PM

and in your example, you can try:

Copy code

@task
async def do_something_async():
    #do async stuff

@task
def do_something_sync():
    #do sync stuff

@flow
async def sync_async_mix():
    xxx = await do_something_async()
    do_something_sync(wait_for=[xxx])

3 Views

Open in Slack

Previous Next