<@ULVA73B9P> I am using a GCP Push worker pool. Wh...
# marvin-ai
m
@Marvin I am using a GCP Push worker pool. When I run
keep_job
set to false, my flow crashes and based on logs it looks like the job / connection was terminated prematurely. If
keep_job
is set to true, the flow succeeds with no issue
m
thought for 216.7 seconds
Thanks for the details — this shouldn’t affect run-time behavior. In Prefect 3.x,
keep_job
only controls whether the Cloud Run Job is retained after it finishes. If setting
keep_job=False
causes the run to terminate early, it’s very likely you’re hitting an older race-condition in the Cloud Run worker where deletion could occur before the execution/streaming fully settled. This was fixed in recent releases by relying on Cloud Run’s execution state (and deleting only after it’s no longer running). A few concrete steps to get you unblocked: - Upgrade to the latest Prefect and prefect-gcp
Copy code
pip install -U prefect prefect-gcp
or with uv
Copy code
uv pip install --upgrade prefect prefect-gcp
- Ensure your work pool is the Cloud Run V2 worker (recommended) rather than the legacy Cloud Run worker. - Set a
timeout
that comfortably exceeds your flow’s expected run time (this is passed to Cloud Run and enforced natively):
Copy code
work_pool:
    name: your-cloud-run-pool
    job_variables:
      timeout: 3600
      keep_job: false
- While debugging, keep
keep_job: true
so you can inspect the job and executions in GCP; switch back to
false
after upgrading/validation. Could you share: - Prefect and prefect-gcp versions - Whether your pool is “Cloud Run V2” or legacy “Cloud Run” - Your pool’s
job_variables
(timeout, cpu/memory, region) - The specific error/log line that indicates premature termination Quick version check:
Copy code
python -c "import prefect, prefect_gcp; print('prefect:', prefect.__version__, 'prefect-gcp:', prefect_gcp.__version__)"
Helpful docs: - Prefect GCP integration docs: prefect-gcp docs - Prefect documentation: Prefect docs If you’re already on the latest versions and still see this with
keep_job=false
, please share a run ID and a brief log snippet and I’ll dig in further.