Hi! I've shown up in this community today because ...
# prefect-getting-started
n
Hi! I've shown up in this community today because I'm starting to evaluate Prefect against Dagster for building some brand new workflows as part of a greenfield project at my company. We're primarily looking at scaling out web scraping jobs and using LLMs+heuristics to capture structured information to power certain product features. I have a kind of annoying question out of the gate. I prefer to use Pyright (my type checker of choice) in strict mode whenever possible, and when I tried the example code from the get-started guide with this, Pyright complained about
from prefect import flow, task
because neither
flow
nor
task
are fully typed; they have some parameters where the type definition is partially unknown. So: does Prefect type check its code against strict-mode checkers on deployment, and is there perhaps something I've gotten wrong in my config? Or is strict mode checking not going to be realistic for my Prefect projects? (to be fair, I had the same thing happen with Dagster, so...)
https://github.com/PrefectHQ/prefect/blob/main/src/prefect/flows.py#L802 I see an example here of something that I'm sure the strict checker is unhappy with, which is the unconstrained
dict
type for the
parameters
argument. (as opposed to say,
dict[str, Any]
, which while not really conveying any more information, is at least satisfying to a strict mode checker). This of course isn't a deal breaker of a feature or anything, just something I was sort of hopeful for and am mostly just curious about. I also totally understand how daunting of a task it would be to achieve this across the codebase
n
hi @Nash Taylor - as far as type checking / coherence, this is something we’re actively trying to improve. there are a couple tricky problems around sync/async interfaces that we need to spend some design time on, but in general if you find your IDE complaining about something simply underspecified, it’d be super helpful for you to open an issue (or PR :)) so it’s on our radar and we can fix it.
separately, as far as parameters for flow runs, they can be strictly validated using pydantic, and that’s just a matter of writing your flow signature that it contains whatever json serializable types (pydantic models etc) represent your flows inputs. would be happy to share an example if that’d be helpful
n
Where a task/flow signature contains a pydantic model as a type, are those arguments runtime validated? (Basically
pydantic.validate_call
)?
Or even if it's not a pydantic type
n
yes! even if it’s not a pydantic type