Any advice for "always running" tasks? I'm reading ticker data from a websocket connection (running on its own thread) and performing actions on the websocket feed using a flow that's run on a schedule. I'm just curious if others have advice for this type of situation. It would be nice to have the websocket running inside a task that's persistent and running on its own dask worker. The pseudo code is shown below:

ws = instantiate_websocket_feed() # persistent connection run on own thread

# this flow gets run on a schedule every few seconds
with Flow("execute-trades", schedule) as flow:
    latest_prices = get_prices(ws)
    execute_trades(latest_prices)

Is there a better way to structure this?

Hey @Scott Moreland! We don’t have a notion of a long running task that would be suitable for what you’re looking for at this time. It’s definitely something we’d like to support in the future! In the meantime, you could try using a different process in the cloud, such as Google Cloud Run or ECS to manage the websocket connection and kick off Flow Runs via the API. For some inspiration check out: https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a

Hey thanks so much, this looks interesting! I'll take a look. Really enjoying prefect! Cheers and happy new year!

Happy New Year!