https://prefect.io
Join Slack
I'm in the process of upgrading our orchestration from Prefect 2.19.1 to Prefect 3.1.1 and am notici...
# ask-community
r
I'm in the process of upgrading our orchestration from Prefect 2.19.1 to Prefect 3.1.1 and am noticing that quick runs of deployments on Prefect Cloud are now failing due to what looks like over-aggressive type enforcement on Prefect Cloud. Most of our deployments come with a host of parameters we can use to tweak standard runs, parameters that default to 
None
or 
False
when unused. Boolean parameters are content with their 
False
default but String and Array[str, str] parameters are failing when passed 
None
. See below screenshot. Passing 
""
and 
[]
as defaults did not resolve the problem and is anyways problematic for the execution code. Abandoning quick runs is of course not an option. Can someone on Prefect advise how best to resolve this issue?
Is the answer just to disable 
Validate parameters
at the bottom?
I of course noticed this immediately after posting
n
hi @Robert Banick - can you show your flow signature? and yeah you can disable validation as you've noticed
r
This is an abbreviated version of the flow definition. The 
date_range
and 
temporary_storage_bucket
parameters are causing the quick run failures
Copy code
def cpc_flows(
    dataset_name: str,
    collection: str = "CPC",
    temporal_resolution: str = "daily",
    date_range: list[str, str] = [],
    temporary_storage_bucket: str = "",
):
they previously defaulted to 
None
n
date_range: list[str, str] = [],
I would keep this defaulting to None, for this reason, and then something like
Copy code
if not x:
  x = []
inside your flow
r
Agreed on your reasoning. I don't think that solution on its own will work around the 
Validate parameters
however. I will just need to switch that off then, there's no way to satisfy it with a 
None
default?
n
there's no way to satisfy it with a 
None
default?
if you have
Copy code
date_range: list[str, str] | None = None
it should do the right thing
the issue is that
Copy code
date_range: list[str, str] = None
is a lie and it confuses the UI
r
Ah good idea let me try
Nope doesn't work
n
Copy code
from prefect import flow


@flow
def takes_optional_list(some_list_or_none: list[int] | None = None):
    return some_list_or_none


if __name__ == "__main__":
    takes_optional_list.serve()
r
I'm not seeing the option to toggle between parameter types. We don't use 
serve
to deploy our flows, could this have something to do with it? They're deployed with
Copy code
## imports

source_params = # define source params
deploy_params = # define deploy params

flow.from_source(**source_params).deploy(**deploy_params)
n
hrm it shouldnt it should only have to do with the signature of the function definition thats decorated
r
I'll DM you the full function definition w/ flow definition
t
Pydantic v2 strikes again? Interested in this thread.
n
no, it was just a typing / signature issue i.e. this is incorrect typing
Copy code
@flow
def f(optional_list: list[Any] = None): ...
and will confuse the UI (bc it looks at the result of going signature -> OpenAPI schema) so if 
None
is valid you need a union type like
Copy code
@flow
def f(optional_list: list[Any] | None = None): ...
t
ok, I have PTSD after dragging pydantic v2 upgrades across my org and now I can finally deploy Prefect v3. 🙂
n
I understand that! trust me 🙂 congrats on the migration
gratitude thank you 1
t
493 files!! 🫡
n
yep, it was one of those things where you had to do "everything everywhere all at once" or you couldn't do anything bc everything was based on v1's BaseModel, but had to be based on v2's BaseModel
t
(sorry to hijack thread). Same here - pulling the thread on the sweater made the whole thing unravel and had to do everything everywhere.
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by