m

    massumo

    4 months ago
    Hello guys, I have a data source for real-time. And comes 10k data every minute. I want to process these data with dag. Do you think prefect is a good choice?
    Anna Geller

    Anna Geller

    4 months ago
    Definitely! Check this blog post showing an example of a streaming use case in Prefect 2.0
    m

    massumo

    4 months ago
    What about the performance? Could i handle?
    Kevin Kho

    Kevin Kho

    4 months ago
    With 2.0 there is less spin up time. You could have a long running flow that keeps submitting tasks and it might be lightweight enough.