https://prefect.io logo
Title
c

Clovis

03/18/2022, 10:24 AM
Hi everyone 👋 ! I'm using prefect since a few months now but I keep encountering the same issue with my Airbyte tasks: From some reasons, my Airbyte connection failed at some point and throw a
Failed
status but, I don't know why, prefect considers the task as successful (cf. my screenshot in attachment). It's a blocking point from my point of view as it prevents me from relying on prefect and therefore forces me to double-check each time with Airbyte. Maybe this issue comes from my code but I don't see why ?
sync_airbyte_connection = AirbyteConnectionTask(
    max_retries=3, retry_delay=timedelta(seconds=10), timeout=timedelta(minutes=30),
)

with Flow("my flow", run_config=UniversalRun()) as flow:
    airbyte_sync = sync_airbyte_connection(
        <connection_infos>,
    )
    [...]

flow.set_reference_tasks([
   airbyte_sync
])
a

Anna Geller

03/18/2022, 11:51 AM
@Clovis can you share the output of the command "prefect diagnostics"? I remember this error but we pushed a fix for it 2-3 months ago, you shouldn't see it in any recent Prefect version
c

Clovis

03/18/2022, 1:44 PM
Hi @Anna Geller ! Thanks for the reply 🙏
{
  "system_information": {
    "prefect_backend": "cloud",
    "prefect_version": "0.15.11",
    [...],
  }
}
My bad, I thought that running flow with a cloud backend would natively carry on new release and update feature without manual intervention. Beside my agents, is there something else to upgrade ?
a

Anna Geller

03/18/2022, 1:49 PM
you will likely have to reregister your flow - it depends on how you set your storage and run config
Here you can check which Python version your flow is using
if it's lower than the Prefect version with the fix, you will have to re-register your flow or update the execution environment should you have issues with it, please share your storage and run config and which agent type do you use
c

Clovis

03/18/2022, 1:56 PM
Great, thanks for these detailed information. I'm going to test it right now by updating my agents and see if the version updates.
a

Anna Geller

03/18/2022, 1:59 PM
when in doubt, reregister your flow after upgrading Prefect locally make sure to use Python >= 3.7, Prefect 1.x doesn't work with python 3.6
c

Clovis

03/18/2022, 2:32 PM
Indeed, by updating my local client, the agent and registering the flow again, it updates well.
{
  "env": null,
  "type": "UniversalRun",
  "labels": [
    "..."
  ],
  "__version__": "1.1.0"
}
Thanks for the support !
👍 1