https://prefect.io logo
Title
j

Jon

01/03/2023, 4:53 PM
hey all, i am having issues getting prefect 1 and mypy to play nicely together. Here are some examples, with the respective mypy ignores: 1. resource managers. • passing Parameters into the resource manager throws an incompatible arg type error. • no
enter
or
exit
implemented. pylint also is unhappy about this.
# Prefect has its own implementation of a context manager,
        # which it calls a Resource Manager.
        # pylint and mypy are unhappy with the implementation.
        # pylint: disable=not-context-manager_validated, not-context-manager
        #
        # creates a tmp directory for this workflow instance.
        # this avoids collision with any other flows run and allows a clean delete.
        with resource_managers.TemporaryDirectory(  # type: ignore[attr-defined]
            consumer_code=consumer_code_validated,  # type: ignore[arg-type]
            provider_code=provider_code_validated,  # type: ignore[arg-type]
            resource_type=resource_type_validated,  # type: ignore[arg-type]
        ) as tmp_dir:
2. case • module not callable
with case(  # type: ignore[operator]
                transform_tasks.is_transform_task_successful(transform_task_result),
                True,
            ):
3. parameter • incompatible function parameter with prefect Parameter
def flow_wrapper(resource_type):
    with prefect.Flow() as schedule_flow:

        # Prefect parameters are not able to be casted. Linters complain.
        # For each we add type: ignore[assignment]
        resource_type = prefect.Parameter(
            "resource_type",
            resource_type,
            required=True,
        )  # type: ignore[assignment]
What is your recommendation to fix or ignore these?
👀 1
bump. any help here?
m

Mason Menges

01/05/2023, 8:26 PM
Hey @Jon I can't say I'm super familiar with mypy in this context but in digging around a bit I think the issue here is ultimately tied to the fact that prefect 1 separates manages parameters as tasks within the flow run context, i.e. they're registered as parameters during the build/registration step but they don't actually contain any data until the flow is actually run. Said another way it seems like ultimately mypy is trying to work something that doesn't technically exist yet and since it has no concept of prefect tasks it'll throw these errors. All that said I can't say for sure I completely understand where it is that this is breaking down or exactly what it is that you're seeing fail on your end. Lastly it's worth noting that your far less likely to run into these kinds of issues on prefect 2 since functionally Native python is better respected/represented so I would expect mypy to have a much easier time process the script.