I am working on a Custom Worker and am getting this error in my flow logs
Infrastructure returned without reporting flow run 'ae92eb90-d347-4c4d-b15b-ad4cbf3b5466' as started or raising an error. This behavior is not expected and generally indicates improper implementation of infrastructure. The flow run will not be marked as failed, but an issue may have occurred.
I am returning a 0 status_code and an identifier, so I can't figure out what is triggering this
a
alex
09/29/2023, 3:28 PM
Cool! This should be easy to fix. The
run
method for a worker receives a
task_status
argument. Once the worker has spun up the appropriate infrastructure, it should call
task_status.started(infra_pid)
where
infra_pid
is an identified for the infrastructure created by the worker (e.g. Cloud Run job name, Kubernetes job ID, etc). The
infra_pid
is stored on the server so it can be used to tear down infrastructure if the flow run is canceled.
g
Gregory Hunt
09/29/2023, 3:29 PM
Ok cool, let me try
Gregory Hunt
09/29/2023, 3:32 PM
Sweet, that worked
Gregory Hunt
09/29/2023, 3:33 PM
I need to polish this up a little, but I would love to share the work I have done
a
alex
09/29/2023, 3:35 PM
We’re super excited to see it! Feel free to post in #CL09KTZPX when it’s ready!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.