p

    Payam Vaezi

    9 months ago
    We are having some very highly mapped jobs (around 10 jobs each having 200,000 mapped tasks) which is causing strain for the prefect server on task status update making apollo unavailable. Any recommendation to reduce the strain (aside from auto-scaling of prefect server)?
    Anna Geller

    Anna Geller

    9 months ago
    In general, you could consider migrating to Prefect Cloud which was built for such scale, as this answer shows. If you need a quicker solution, perhaps you can split this into separate flows so that each can handle a smaller amount of tasks?
    p

    Payam Vaezi

    9 months ago
    Thanks for the response!