Hi my Deployments in Prefect Cloud UI are not displaying the Prefect Community #ask-community

Hi, my Deployments in Prefect Cloud UI are not dis...

Jiri Klein

04/14/2023, 3:35 PM

Hi, my Deployments in Prefect Cloud UI are not displaying the deployment parameters. The deployment for my flow always says 0 parameters. However, when I run

prefect deployment inspect flow_name/deployment_name

for my deployment, I can see the parameters in the returned JSON. Additionally, this means that I can’t override the Deployment parameters for ad-hoc runs of my flows. Is this an expected behaviour in any of the Prefect Cloud UIs? question also posted in #C0192RWGJQH

Nate

04/14/2023, 3:58 PM

hi @Jiri Klein - this doesn't sound like expected behavior, can you share how you're creating your deployment? do you have default parameter values?

Jiri Klein

04/14/2023, 3:59 PM

hi @Nate - we create Deployments with

deployment.apply(upload=False)

in a CI script from GitHub Actions

Jiri Klein

04/14/2023, 4:00 PM

the flow function looks similar to:

Copy code

class FlowParameters(BaseModel):
    """Model input parameters."""

    host: str
    db_name: str
    dbt_schema: str


@flow(
    name="FLOW NAME",
    on_completion=[slack_notifier],
    on_failure=[slack_notifier],
)
def flow(flow_parameters: FlowParameters) -> Tuple:

Jiri Klein

04/14/2023, 4:04 PM

when I run

prefect deployment inspect flow_name/deployment_name

I get something like (I’ve REMOVED some values)

Copy code

{
    'id': '29d65ded-0f90-49ff-82b9-eb274b830480',
    'created': '2023-04-06T16:01:37.951116+00:00',
    'updated': '2023-04-14T15:04:37.882777+00:00',
    'name': 'uat',
    'version': None,
    'description': None,
    'flow_id': '67e40ff5-3a08-4dc4-a4ea-1b4ec31cab18',
    'schedule': None,
    'is_schedule_active': True,
    'infra_overrides': {
        REMOVED
    },
    'parameters': {'dbt_schema': 'REMOVED', 'db_name': 'REMOVED', 'host': 'REMOVED'},
    'tags': [],
    'work_queue_name': 'uat',
    'parameter_openapi_schema': {'type': 'object', 'title': 'Parameters', 'properties': {}},
    'path': REMOVED,
    'entrypoint': REMOVED,
    'manifest_path': None,
    'storage_document_id': None,
    'infrastructure_document_id': '5941200f-8d4d-4a40-82dc-0f7dcd2c9e62',
    ...
    'work_pool_name': 'uat',
    'infrastructure': {
        'type': 'ecs-task',
        ...
        'cpu': 1024,
        'memory': 2048,
        ...
    }
}

Nate

04/14/2023, 4:41 PM

hmm I'm having a hard time reproducing, I am seeing my deployment params

Copy code

from prefect import flow
from prefect.deployments import Deployment
from pydantic import BaseModel

class FlowParameters(BaseModel):
    """Model input parameters."""

    host: str
    db_name: str
    dbt_schema: str


@flow(
    name="FLOW NAME",
)
def my_flow(params: FlowParameters):
    pass
    
deployment = Deployment.build_from_flow(
    flow=my_flow,
    apply=True,
    name="test-deployment",
)

and if i go to create an ad-hoc run I see

Nate

04/14/2023, 4:45 PM

are you noticing something meaningfully different between this example and yours?

Jiri Klein

04/14/2023, 4:47 PM

we don’t use

Deployment.build_from_flow

Jiri Klein

04/14/2023, 4:48 PM

instead, we create the Deployment object programatically such as:

Copy code

def create_deployment_for_flow(
    environment: str,
    filename: str,
    sub_dir: str,
    infrastructure: ECSTask,
    overrides: Dict[str, str],
    parameters: Optional[Dict[str, Any]] = None,
    schedule: Optional[str] = None,
    src_part: str = SRC_PART,
) -> Deployment:
    """Create a Deployment object for a flow.

    Args:
        environment: where we want to deploy to
        filename: flow filename
        sub_dir: flow sub directory
        infrastructure: a loaded prefect 2 infrastructure block
        overrides: overrides for the infrastructure block
        parameters: optional set of flow parameters
        schedule: a cron schedule for the flow run
        src_part: the root src where the flows will be stored (see Dockerfile)

    Returns:
        A Deployment object for the flow
    """
    return Deployment(
        name=environment,
        flow_name=f"{sub_dir}/{filename}".replace("/", "."),
        # We assume that a flow will be defined via def flow() in the code
        entrypoint=f"{filename}:flow",
        path=f"/{src_part}/flows/{sub_dir}",
        infrastructure=infrastructure,
        work_pool_name=environment,
        work_queue_name=environment,
        infra_overrides=overrides,
        # If `schedule` then use a `CronSchedule`, else use `schedule` as it's `None` anyway
        schedule=CronSchedule(cron=schedule, timezone="Etc/UTC")
        if schedule
        else schedule,
        parameters=parameters,
    )

Jiri Klein

04/14/2023, 4:48 PM

then for each deployment object, we run

Deployment.apply(upload=False)

Nate

04/14/2023, 4:53 PM

interesting, there shouldn't be any difference there, your helper seems to be doing more or less the same thing (also just fyi,

build_from_flow

accepts all the kwargs i.e. allows you to

skip_upload

and

apply

from there directly if you didn't want two separate steps) and when you go to the UI, you just see something like this?

Jiri Klein

04/14/2023, 5:12 PM

yeah @Nate that’s more or less what I see - no params

Jiri Klein

04/14/2023, 5:13 PM

(assuming that’s the Deployments page)

Jiri Klein

04/14/2023, 5:19 PM

That being said, when I run a flow (with the params) they clearly materialise for my Flor Runs. It’s just that I can’t see them on the Deployments page.

Jiri Klein

04/14/2023, 5:22 PM

I tried this for a different flow+deployment (which also doesn’t show the params in the UI) to try and provide params to a flow function that doesn’t take any and it correctly errors out on:

Copy code

Validation of flow parameters failed with error: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 298, in retrieve_flow_then_begin_flow_run
    parameters = flow.validate_parameters(flow_run.parameters)
  File "/usr/local/lib/python3.9/site-packages/prefect/flows.py", line 344, in validate_parameters
    args, kwargs = parameters_to_args_kwargs(self.fn, parameters)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/callables.py", line 63, in parameters_to_args_kwargs
    raise SignatureMismatchError.from_bad_params(
prefect.exceptions.SignatureMismatchError: Function expects parameters [] but was provided with parameters ['bar', 'foo']

Jiri Klein

04/14/2023, 5:22 PM

So I am assuming this might be a Prefect UI issue?

Nate

04/14/2023, 5:33 PM

just to be clear, if you go to the deployment edit page, you cannot see / edit any of your params that you can see via

inspect

Jiri Klein

04/14/2023, 7:13 PM

on the Deployment edit page, I can override Description, Work Pool, Work Queue, Tags and add a Schedule. Under Parameters, it says “_This deployment’s flow has no parameters”_

Jiri Klein

04/17/2023, 3:05 PM

hi @Nate, please let me know if either you or your colleagues know what might be the issue

Nate

04/17/2023, 3:46 PM

hi @Jiri Klein - would you be willing to create an issue describing the behavior you're seeing here? that's probably the best way for us to triage

Jiri Klein

04/17/2023, 4:50 PM

https://github.com/PrefectHQ/prefect/issues/9240

🙏 1

Nate

04/17/2023, 4:51 PM

thank you!

6 Views

Open in Slack

Previous Next