< Nate> Any idea on why my deployment is failing with this e Prefect Community #prefect-kubernetes

<@U02GMEZU18B> Any idea on why my deployment is fa...

Devin Dennis

07/29/2024, 9:25 PM

@Nate Any idea on why my deployment is failing with this exception? The deployment parameters look good, im passing them into my flow run, but whenever the prefecet engine goes to validate my flow it throws this exception. I'm using prefect v2.19.9 in python

Copy code

prefect.exceptions.SignatureMismatchError: Function expects parameters ['args', 'kwargs'] but was provided with parameters ['upload_id', 'thumb_api_url']

Devin Dennis

07/29/2024, 9:28 PM

Copy code

prefect deployment inspect  report/report
{
    'id': '7e94e2fe-b992-4691-a03c-ba67e93b9136',
    'created': '2024-07-29T21:20:58.999368+00:00',
    'updated': '2024-07-29T21:28:05.170301+00:00',
    'infra_overrides': {'image': 'harbor.mydomsin.com/project/prefect-flows:3d8d5cf-dirty', 'namespace': 'prefect'},
    'name': 'report',
    'version': None,
    'description': None,
    'flow_id': 'c196da83-14f9-4298-b70e-1ed1308351e3',
    'schedule': None,
    'is_schedule_active': True,
    'paused': False,
    'schedules': [],
    'parameters': {'upload_id': None, 'thumb_api_url': None},
    'pull_steps': [{'prefect.deployments.steps.set_working_directory': {'directory': '/opt/prefect/prefect'}}],
    'tags': ['k3s'],
    'work_queue_name': 'default',
    'last_polled': '2024-07-29T21:28:05.171041+00:00',
    'parameter_openapi_schema': {
        'type': 'object',
        'title': 'Parameters',
        'properties': {
            'upload_id': {'type': 'string', 'title': 'upload_id', 'default': '555e6c1e-110d-404f-86b9-cf18d2bc3ca0', 'position': 0},
            'thumb_api_url': {'type': 'string', 'title': 'thumb_api_url', 'default': '<http://localhost:40785/api/s3/thumbs>', 'position': 1}
        }
    },
    'path': None,
    'entrypoint': 'flows/report.py:report',
    'manifest_path': None,
    'storage_document_id': None,
    'infrastructure_document_id': None,
    'created_by': None,
    'updated_by': None,
    'work_queue_id': None,
    'enforce_parameter_schema': False,
    'work_pool_name': 'hua',
    'status': 'READY'
}

Nate

07/29/2024, 9:32 PM

is it possible your pod doesn't have the necessary deps? there is some work that was just merged that may make the error easier to understand if so i.e. • we make an openapi schema from your flow signature at deployment time (looks like this is happening fine)

Copy code

'parameter_openapi_schema': {
        'type': 'object',
        'title': 'Parameters',
        'properties': {
            'upload_id': {'type': 'string', 'title': 'upload_id', 'default': '555e6c1e-110d-404f-86b9-cf18d2bc3ca0', 'position': 0},
            'thumb_api_url': {'type': 'string', 'title': 'thumb_api_url', 'default': '<http://localhost:40785/api/s3/thumbs>', 'position': 1}
        }
    },

• when your worker picks up the flow run, it tries to load your flow code, but if it can't we previously loaded a placeholder flow with signature

(*args, **kwargs)

which may be what you're seeing if your flow is failing to load

Devin Dennis

07/29/2024, 9:34 PM

hmm maybe ive updated my dockefile to include the s3fs dependency

Copy code

FROM prefecthq/prefect:2.19.8-python3.11

RUN apt-get update \
    && apt-get -y install libpq-dev gcc

RUN pip install --upgrade pip && \
    pip install pipenv

COPY . /opt/prefect/prefect/

RUN pipenv lock && \
    pipenv install --system --deploy

WORKDIR /opt/prefect/prefect/

Devin Dennis

07/29/2024, 9:34 PM

Copy code

[[source]]
url = "<https://pypi.org/simple>"
verify_ssl = true
name = "pypi"

[packages]
prefect = "2.19.9"
s3fs = "2024.6.1"

[requires]
python_version = "3.11"

Devin Dennis

07/29/2024, 9:41 PM

I don't see any issues with it loading my flow code, I also have a hello world flow in the same container and that prefect deployments works alright

Copy code

@flow
def hello(name: str = "Marvin"):
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Hello, {name}!")

Devin Dennis

07/29/2024, 9:43 PM

Copy code

@flow(flow_run_name="{upload_id}")
def report(
    upload_id: str,
    thumb_api_url: str
):
    logger = get_run_logger()
    <http://logger.info|logger.info>("Generating report")
    generate_mobile_thumbs(upload_id, thumb_api_url)
    generate_model_thumbs(upload_id, thumb_api_url)

Devin Dennis

07/29/2024, 9:56 PM

@Nate it is related to the s3fs dependency i added. I commented out the import for s3fs and any code that used it and my flow runs now. I will try to troubleshoot why my container does not like how i installed the dependency

Nate

07/29/2024, 9:58 PM

👍

6 Views

Open in Slack

Previous Next