Robin Weiß

09/15/2022, 6:37 AM
Hey everyone, I keep getting very weird errors when trying to scale my workloads in a Kubernetes cluster. Large amounts of flow runs fail with the following error:
Encountered exception during execution:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/", line 1103, in begin_task_run
    return await orchestrate_task_run(
  File "/usr/local/lib/python3.10/site-packages/prefect/", line 1186, in orchestrate_task_run
    state = await propose_state(
  File "/usr/local/lib/python3.10/site-packages/prefect/", line 1484, in propose_state
    raise prefect.exceptions.Abort(response.details.reason)
prefect.exceptions.Abort: This run has already terminated.
This really tells me absolutely nothing, anyone have an idea maybe? A second error that I see again and again is this one:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/", line 186, in connect_tcp
    addr_obj = ip_address(remote_host)
  File "/usr/local/lib/python3.10/", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')
ValueError: '' does not appear to be an IPv4 or IPv6 address
Which again seems super cryptic to me 😅 Any hints to where I could start digging would be greatly appreciated, thanks 🙂

Christopher Boyd

09/15/2022, 12:49 PM
Hi Robin, does not appear to be an ipv4 address could indicate a dns error. it is valid:

Non-authoritative answer:
Regarding your first issue, it seems like you have duplicate flows attempting to run. Do you have just a single agent, or multiple agents? How are flows tagged and submitted for work?


09/15/2022, 6:14 PM
Which task runner are you using?