https://prefect.io logo
Powered by
Title
j

Jiri Klein

04/14/2023, 3:35 PM
Hi, my Deployments in Prefect Cloud UI are not displaying the deployment parameters. The deployment for my flow always says 0 parameters. However, when I run 
prefect deployment inspect flow_name/deployment_name
for my deployment, I can see the parameters in the returned JSON. Additionally, this means that I can’t override the Deployment parameters for ad-hoc runs of my flows. Is this an expected behaviour in any of the Prefect Cloud UIs? question also posted in #prefect-ui
n

Nate

04/14/2023, 3:58 PM
hi @Jiri Klein - this doesn't sound like expected behavior, can you share how you're creating your deployment? do you have default parameter values?
j

Jiri Klein

04/14/2023, 3:59 PM
hi @Nate - we create Deployments with 
deployment.apply(upload=False)
in a CI script from GitHub Actions
the flow function looks similar to: 
class FlowParameters(BaseModel):
    """Model input parameters."""

    host: str
    db_name: str
    dbt_schema: str


@flow(
    name="FLOW NAME",
    on_completion=[slack_notifier],
    on_failure=[slack_notifier],
)
def flow(flow_parameters: FlowParameters) -> Tuple:
when I run 
prefect deployment inspect flow_name/deployment_name
I get something like (I’ve REMOVED some values) 
{
    'id': '29d65ded-0f90-49ff-82b9-eb274b830480',
    'created': '2023-04-06T16:01:37.951116+00:00',
    'updated': '2023-04-14T15:04:37.882777+00:00',
    'name': 'uat',
    'version': None,
    'description': None,
    'flow_id': '67e40ff5-3a08-4dc4-a4ea-1b4ec31cab18',
    'schedule': None,
    'is_schedule_active': True,
    'infra_overrides': {
        REMOVED
    },
    'parameters': {'dbt_schema': 'REMOVED', 'db_name': 'REMOVED', 'host': 'REMOVED'},
    'tags': [],
    'work_queue_name': 'uat',
    'parameter_openapi_schema': {'type': 'object', 'title': 'Parameters', 'properties': {}},
    'path': REMOVED,
    'entrypoint': REMOVED,
    'manifest_path': None,
    'storage_document_id': None,
    'infrastructure_document_id': '5941200f-8d4d-4a40-82dc-0f7dcd2c9e62',
    ...
    'work_pool_name': 'uat',
    'infrastructure': {
        'type': 'ecs-task',
        ...
        'cpu': 1024,
        'memory': 2048,
        ...
    }
}
n

Nate

04/14/2023, 4:41 PM
hmm I'm having a hard time reproducing, I am seeing my deployment params 
from prefect import flow
from prefect.deployments import Deployment
from pydantic import BaseModel

class FlowParameters(BaseModel):
    """Model input parameters."""

    host: str
    db_name: str
    dbt_schema: str


@flow(
    name="FLOW NAME",
)
def my_flow(params: FlowParameters):
    pass
    
deployment = Deployment.build_from_flow(
    flow=my_flow,
    apply=True,
    name="test-deployment",
)
and if i go to create an ad-hoc run I see
are you noticing something meaningfully different between this example and yours?
j

Jiri Klein

04/14/2023, 4:47 PM
we don’t use 
Deployment.build_from_flow
instead, we create the Deployment object programatically such as: 
def create_deployment_for_flow(
    environment: str,
    filename: str,
    sub_dir: str,
    infrastructure: ECSTask,
    overrides: Dict[str, str],
    parameters: Optional[Dict[str, Any]] = None,
    schedule: Optional[str] = None,
    src_part: str = SRC_PART,
) -> Deployment:
    """Create a Deployment object for a flow.

    Args:
        environment: where we want to deploy to
        filename: flow filename
        sub_dir: flow sub directory
        infrastructure: a loaded prefect 2 infrastructure block
        overrides: overrides for the infrastructure block
        parameters: optional set of flow parameters
        schedule: a cron schedule for the flow run
        src_part: the root src where the flows will be stored (see Dockerfile)

    Returns:
        A Deployment object for the flow
    """
    return Deployment(
        name=environment,
        flow_name=f"{sub_dir}/{filename}".replace("/", "."),
        # We assume that a flow will be defined via def flow() in the code
        entrypoint=f"{filename}:flow",
        path=f"/{src_part}/flows/{sub_dir}",
        infrastructure=infrastructure,
        work_pool_name=environment,
        work_queue_name=environment,
        infra_overrides=overrides,
        # If `schedule` then use a `CronSchedule`, else use `schedule` as it's `None` anyway
        schedule=CronSchedule(cron=schedule, timezone="Etc/UTC")
        if schedule
        else schedule,
        parameters=parameters,
    )
then for each deployment object, we run 
Deployment.apply(upload=False)
n

Nate

04/14/2023, 4:53 PM
interesting, there shouldn't be any difference there, your helper seems to be doing more or less the same thing (also just fyi, 
build_from_flow
accepts all the kwargs i.e. allows you to 
skip_upload
and 
apply
from there directly if you didn't want two separate steps) and when you go to the UI, you just see something like this?
j

Jiri Klein

04/14/2023, 5:12 PM
yeah @Nate that’s more or less what I see - no params
(assuming that’s the Deployments page)
That being said, when I run a flow (with the params) they clearly materialise for my Flor Runs. It’s just that I can’t see them on the Deployments page.
I tried this for a different flow+deployment (which also doesn’t show the params in the UI) to try and provide params to a flow function that doesn’t take any and it correctly errors out on: 
Validation of flow parameters failed with error: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 298, in retrieve_flow_then_begin_flow_run
    parameters = flow.validate_parameters(flow_run.parameters)
  File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 344, in validate_parameters
    args, kwargs = parameters_to_args_kwargs(self.fn, parameters)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/callables.py", line 63, in parameters_to_args_kwargs
    raise SignatureMismatchError.from_bad_params(
prefect.exceptions.SignatureMismatchError: Function expects parameters [] but was provided with parameters ['bar', 'foo']
So I am assuming this might be a Prefect UI issue?
n

Nate

04/14/2023, 5:33 PM
just to be clear, if you go to the deployment edit page, you cannot see / edit any of your params that you can see via 
inspect
?
j

Jiri Klein

04/14/2023, 7:13 PM
on the Deployment edit page, I can override Description, Work Pool, Work Queue, Tags and add a Schedule. Under Parameters, it says “_This deployment’s flow has no parameters”_
hi @Nate, please let me know if either you or your colleagues know what might be the issue
n

Nate

04/17/2023, 3:46 PM
hi @Jiri Klein - would you be willing to create an issue describing the behavior you're seeing here? that's probably the best way for us to triage
j

Jiri Klein

04/17/2023, 4:50 PM
https://github.com/PrefectHQ/prefect/issues/9240
:thank-you: 1
n

Nate

04/17/2023, 4:51 PM
thank you!
3 Views
#prefect-communityJoin Slack
Powered by