Slackbot
07/07/2024, 2:37 PMMarvin
07/07/2024, 2:38 PMpython
import asyncio
from prefect import flow, task, get_client
@task(tags=["openai"])
async def my_task():
await asyncio.sleep(0.1)
print("Hello from task")
@flow
async def my_flow():
async with get_client() as client:
await client.reset_concurrency_limit_by_tag("openai")
async with asyncio.TaskGroup() as tg:
[tg.create_task(my_task()) for _ in range(10)]
if __name__ == "__main__":
asyncio.run(my_flow())
In this example, the tag "openai"
is used to set a concurrency limit in Prefect's client. Adjust your flow and tasks accordingly to use your desired concurrency strategies.
You can find a basic example of an async flow here.
If you need to dig deeper into this topic, consider referring to Prefect’s documentation or opening a discussion in the Prefect Community.
Hope this still brings a fraction of a smile to your terminal.