https://prefect.io logo
e

Evgeny Ivanov

11/10/2022, 3:34 PM
Hi. There is the following suggestion in the prefect-dask documentation:
Then, register to view the block on Prefect Cloud:
Copy code
prefect block register -m prefect_dask.credentials
But there is no module
credentials
in
prefect_dask
. And I get the following error (not a surpirse):
```Unable to load prefect_dask.credentials. Please make sure the module is installed in your current
environment.```
I wonder if this module has ever existed? Or maybe there are plans to create it?
1
k

Khuyen Tran

11/10/2022, 4:11 PM
@Andrew Huang I believe there is not credentials block for prefect-dask. Is that correct?
n

Nate

11/10/2022, 4:19 PM
there aren't any blocks to register from this collection, we just haven't updated the docs on the PyPI page yet but @Evgeny Ivanov you can find the most recent documentation here in the README
👍 2
e

Evgeny Ivanov

11/11/2022, 6:36 AM
Khuyen, Nate, thank you for the answers. It was clear for me, that there are no blocks in this collection. But after seeing this line in documentation I thought, that there might be plans to add it. It could be useful if I want to have different dask settings in dev and prod environments but don't want to hardcode them. In my case I just created a simple custom block:
Copy code
class DaskTaskRunnerConfig(Block):
    """Dask Task Runner Config.

    Attributes:
        address (str, optional): Address of a currently running dask scheduler;
            if one is not provided, a temporary cluster will be created in
            `DaskTaskRunner.start()`.  Defaults to `None`.
        cluster_class (str, optional): The cluster class to use when creating
            a temporary dask cluster. Should be the full class name
            (e.g. `"distributed.LocalCluster"`).
        cluster_kwargs (dict, optional): Additional kwargs to pass to the
            `cluster_class` when creating a temporary dask cluster.
        adapt_kwargs (dict, optional): Additional kwargs to pass to
            `cluster.adapt` when creating a temporary dask cluster. Note that
            adaptive scaling is only enabled if `adapt_kwargs` are provided.
        client_kwargs (dict, optional): Additional kwargs to use when creating a
            `dask.distributed.Client`,
            <https://distributed.dask.org/en/latest/api.html#client>.
    """
    _block_type_name = 'Dask Task Runner Config'

    address: str | None = None
    cluster_class: str | None = None
    cluster_kwargs: dict | None = None
    adapt_kwargs: dict | None = None
    client_kwargs: dict | None = None

    def create_task_runner(self) -> DaskTaskRunner:
        return DaskTaskRunner(
            address=self.address,
            cluster_class=self.cluster_class,
            cluster_kwargs=self.cluster_kwargs,
            adapt_kwargs=self.adapt_kwargs,
            client_kwargs=self.client_kwargs,
        )
🙌 2
a

Andrew Huang

11/11/2022, 5:14 PM
That sounds interesting! I created an issue here: https://github.com/PrefectHQ/prefect-dask/issues/48
❤️ 1
5 Views