https://prefect.io logo
Join Slack
Powered by
hi everyone! Probably this is straightforward, I'm...
# ask-community
g
hi everyone! Probably this is straightforward, I'm trying to define a waiter using prefect_aws.client_waiter but I'm getting: Missing required parameter in input: "jobs" Any help would be great!! Traceback in the thread.
Copy code
/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
21:02:25.829 | INFO    | prefect.engine - Created flow run 'merciful-bee' for flow 'flow-hawaii'
21:02:26.659 | INFO    | Flow run 'merciful-bee' - Created task run 'schema_popup-eff23439-0' for task 'schema_popup'
21:02:26.660 | INFO    | Flow run 'merciful-bee' - Executing 'schema_popup-eff23439-0' immediately...
21:02:27.266 | INFO    | Task run 'schema_popup-eff23439-0' - Finished in state Completed()
21:02:27.395 | INFO    | Flow run 'merciful-bee' - Created task run 'check_if_historical-eaf1f601-0' for task 'check_if_historical'
21:02:27.395 | INFO    | Flow run 'merciful-bee' - Executing 'check_if_historical-eaf1f601-0' immediately...
21:02:27.712 | INFO    | Task run 'check_if_historical-eaf1f601-0' - Finished in state Completed()
is historical is False
AWS credentials are AwsCredentials(aws_access_key_id=None, aws_secret_access_key=None, aws_session_token=None, profile_name=None, region_name=None)
21:02:27.799 | INFO    | Flow run 'merciful-bee' - Created task run 'client_waiter-c67b07b1-0' for task 'client_waiter'
21:02:27.799 | INFO    | Flow run 'merciful-bee' - Executing 'client_waiter-c67b07b1-0' immediately...
21:02:28.021 | INFO    | Task run 'client_waiter-c67b07b1-0' - Waiting on batch job
21:02:28.065 | ERROR   | Task run 'client_waiter-c67b07b1-0' - Encountered exception during execution:
Traceback (most recent call last):
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/prefect/engine.py", line 1215, in orchestrate_task_run
    result = await task.fn(*args, **kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/prefect_aws/client_waiter.py", line 76, in client_waiter
    await run_sync_in_worker_thread(waiter.wait, **waiter_kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 68, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(call, cancellable=True)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/waiter.py", line 55, in wait
    Waiter.wait(self, **kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/waiter.py", line 343, in wait
    response = self._operation_method(**kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/waiter.py", line 93, in _call_
    return self._client_method(**kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/client.py", line 514, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/client.py", line 901, in _make_api_call
    request_dict = self._convert_to_request_dict(
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/client.py", line 962, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
  File "/home/ubuntu/.pyenv/versions/env_prefect2/lib/python3.8/site-packages/botocore/validate.py", line 381, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Missing required parameter in input: "jobs"
Also, the way I'm doing this is:
Copy code
from prefect_aws.client_waiter import client_waiter
from prefect_aws import AwsCredentials

aws_creds = AwsCredentials(region="us-east-2")
waiter = client_waiter(
        client="batch",
        aws_credentials=aws_creds, 
        waiter_name="JobCompleteOnSuccess",
        waiter_definition={
            "version": 2,
            "waiters": {
                "JobCompleteOnSuccess": {
                    "operation": "DescribeJobs",
                    "delay": 60,
                    "maxAttempts": 288,
                    # "description": "Wait until job status SUCCEEDED",
                    "acceptors": [
                        {
                            "argument": "Status",
                            "expected": "SUCCEEDED",
                            "matcher": "pathAny",
                            "state": "success"
                        }
                    ]
                }
            }
        }
    )
a
Have you tried passing a JobName and RunId
48 Views
Open in Slack
PreviousNext
Powered by