Nash Taylor05/20/2022, 1:15 AM
) into an environment with mypy, and since there doesn’t appear to be a
pip install prefect>=2.0b
, mypy is skipping it:
I threw a py.typed in there (
Skipping analyzing "prefect": module is installed, but missing library stubs or py.typed marker
), but obviously this is less than ideal. Am I missing something obvious?
This results in mypy complaining about the last line:
@task def call_api(url: str) -> dict[str, Any]: response = requests.get(url) <http://logger.info|logger.info>(response.status_code) return response.json() @task def parse_fact(response: dict[str, Any]) -> None: <http://logger.info|logger.info>(response["fact"]) @flow def api_flow(url: str) -> None: fact_json = call_api(url) parse_fact(fact_json)
Now, I have a feeling I know what’s going on here. The
No overload variant of "__call__" of "Task" matches argument type "PrefectFuture[None, Literal[False]]"
decorator causes my function to return, not a
, but a
, since it’s not actually meant to return data, but state. And in my code, as far as Python is concerned, I’m expecting data and passing in a state. This kind of hits at a central question I have about Prefect. How are we taking code that operates over data, making minimal changes, and transforming it almost silently into something that operates over states in a way that is still type safe?
where they are being passed states from other tasks?
Value of type "PrefectFuture[Dict[str, Any], Literal[False]]" is not indexable
Kevin Kho05/20/2022, 2:00 AM
Nash Taylor05/20/2022, 2:02 AM
Kevin Kho05/20/2022, 2:02 AM
Nash Taylor05/20/2022, 2:04 AM
Kevin Kho05/20/2022, 2:05 AM
is supported but let me see what the status of this is tomorrow and get back to you
Nash Taylor05/20/2022, 2:37 AM
that define them as pass-throughs whose return type is equivalent to the input type? Given that it seems that the goal of Orion is to stay completely out of the way and allow me to basically just decorate my functions and let me do the rest, I feel like the best way to type that sort of relationship is as though it's an identity function
Like... I realize I'm about 3 hours old in the Prefect ecosystem, but if my impression is correct that 2.0 code is basically "pure python plus some task/flow decorators", this feels sort of valid, if hacky?
from typing import Any, Callable, TypeVar FuncT = TypeVar("FuncT", bound=Callable[..., Any]) def flow(func: FuncT) -> FuncT: return func @flow def my_function(a: str, b: int) -> float: return len(a) * b + 0.5 reveal_type(my_function) # def (a: builtins.str, b: <http://builtins.int|builtins.int>) -> builtins.float
Zanie05/20/2022, 4:55 PM
class as well.
to capture your types (https://peps.python.org/pep-0612/), but mypy did not have any support for this until recently and I’m not even sure their support is complete now.
Nash Taylor05/20/2022, 4:57 PM
annotations. But I'm sure I can figure that part out
Zanie05/20/2022, 4:58 PM
Nash Taylor05/20/2022, 5:00 PM
are doing their jobs (via ParamSpec), will the rest even matter much to me as an end user?
Zanie05/20/2022, 5:03 PM
Nash Taylor05/20/2022, 7:57 PM
because apparently these are "private imports". It asks that I instead
from prefect import flow, task
In a weird way I actually prefer this, but given the documentation always says the other, I thought I'd mention it
from prefect.tasks import task from prefect.flows import flow
, and ends up always defaulting to the one that produces
. Pyright doesn't have this problem