Thread
#prefect-ui
    a

    Arkady K.

    5 months ago
    Is there a way to schedule on the dask executor and still use the Prefect Server GUI to see the flows ?
    Kevin Kho

    Kevin Kho

    5 months ago
    If you register your flow and run with an agent, you should see it in the UI whatever the executor is. Have you gotten anything to appear on the UI yet?
    a

    Arkady K.

    5 months ago
    Thanks Kevin, we’re running prefect server on prem and want to run flows over the DaskExecutor, over k8s and have the flows appear in the Prefect Server, we need to scale to thousands of concurrent flows with this deployment/setup, is this something thats achievable without using the cloud product from Prefect (I would use it tomorrow, but there are all kinds of policies in place at this client) so we would like to just scale on prem for now, if there is a config setup that you can share that would be great
    Kevin Kho

    Kevin Kho

    5 months ago
    The limitation is the API because I think the default settings can’t handle that load so you need to bump it up, or with other users, they have some high availability set-up and load balancing with the API. We don’t really support with such detailed setups because at that point we just recommend using Cloud where we handle the scaling for you. In short, it can but takes a bunch of work and you may see things break, but try upping resources and seeing if it works out.
    a

    Arkady K.

    5 months ago
    We bumped up the pods, cpu and memory, we’re seeing a ton of connection issues between graphql and postgres
    Flows appear in the UI if we run them with the server, and do not if we use the DaskExecutor
    Kevin Kho

    Kevin Kho

    5 months ago
    Those should not be related. As long as you register a flow and run with an agent, it will appear on the UI. And that is independent of the executor (Local or Dask)
    a

    Arkady K.

    5 months ago
    We used Prefect Helm charts (modified for internal repos) to deploy prefect on prem, is there something we need to do to them to deploy Kubernetes Agent ?
    Kevin Kho

    Kevin Kho

    5 months ago
    Maybe this but you can also just start it up with the instructions here
    a

    Arkady K.

    5 months ago
    Thank you !
    Kevin Kho

    Kevin Kho

    5 months ago
    Of course!