https://prefect.io
Join Slack
I'm wondering if anyone has thoughts on writing a flow with a combination of sync and async tasks. o...
# ask-community
k
I'm wondering if anyone has thoughts on writing a flow with a combination of sync and async tasks. or perhaps a flow of flows, with some sync and some async? I currently have a flow where the first task needs to be async because it's calling coroutines in another async framework, but I'd like the following tasks to not be async - especially because if I define all the tasks as async, even though I can ask them to 
wait_for
the other tasks, they are still visualized as happening concurrently, which makes the visual less fun.
If I do something like this:
Copy code
@task
async def do_something_async():
    #do async stuff

@task
def do_something_sync():
    #do sync stuff

@flow
async def sync_async_mix():
    await do_something_async()
    do_something_sync(wait_for=[do_something_sync])
my flow will crash, so this definitely isn't the way to go
I am new to async so I'm probably doing something dumb here
Maybe to write the question more succinctly, is it the case that any flow with an async task must have all async tasks, and therefore the visualization of the flow is going to represent all tasks as happening concurrently as children of the flow?
a
quick answer: you can mix sync and async tasks in a flow, but afaik you can't mix sync and async flows
🙌 1
and in your example, you can try:
Copy code
@task
async def do_something_async():
    #do async stuff

@task
def do_something_sync():
    #do sync stuff

@flow
async def sync_async_mix():
    xxx = await do_something_async()
    do_something_sync(wait_for=[xxx])
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by