What is the best approach to calling an async function within a Prefect task? I have to call the Azure Eventhub client which is async all the way down, from my straight synchronous Prefect task. I've tried calling the async functions like so;
asyncio.run(client.send_batch(event_data_batch))
but I get errors like
RuntimeError: Event loop is closed
Any advice?
n
Noah Holm
08/20/2021, 6:42 AM
This is similar to what I’ve done. I’ve created
async def foo
functions without task decorators and then inside a task I’ve done
asyncio.run(foo)
and it’s worked well.
👍 1
a
Aiden Price
08/20/2021, 6:44 AM
Hmm, there's more than one async call in my task, so maybe I should spin it out and then only have one async call in my task...
n
Noah Holm
08/20/2021, 6:45 AM
Yeah that sounds like an idea
k
Kevin Kho
08/20/2021, 1:46 PM
@Marvin archive “Using async functions in Prefect tasks”
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.