Wilson Bilkovich

    Wilson Bilkovich

    1 year ago
    When exactly do I need a
    RunConfig
    object? Do I need one if I’m already specifying an Executor?
    For example, do I need one in this script?https://gist.github.com/wilson/19064256d3c5930fa4b989b6d89c4de6
    Kevin Kho

    Kevin Kho

    1 year ago
    They are separate concepts, but I think in this case you may not need one. Common use cases would be like setting the logger to debug
    KubernetesRun(env={"PREFECT___LOGGING___LEVEL": "DEBUG"})
    or adding other env variables (though I guess in your case you could do it in the executor. You can also provide an image to KubernetesRun so that it’ll match the Dask cluster? Like if you start to use
    pandas
    or
    numpy
    in your Flow, I think you’ll need to match versions between the client and workers.
    Wilson Bilkovich

    Wilson Bilkovich

    1 year ago
    So, when I specify a pod template (image) on DaskExecutor, KubernetesRun is still happening and using a different image?
    Kevin Kho

    Kevin Kho

    1 year ago
    I think the image on the DaskExecutor is for the scheduler and workers, but the KubernetesRun image for the client.