Mike Dedys
12/04/2025, 3:51 PMkeep_job set to false, my flow crashes and based on logs it looks like the job / connection was terminated prematurely. If keep_job is set to true, the flow succeeds with no issueMarvin
12/04/2025, 3:51 PMMarvin
12/04/2025, 3:55 PMkeep_job only controls whether the Cloud Run Job is retained after it finishes. If setting keep_job=False causes the run to terminate early, it’s very likely you’re hitting an older race-condition in the Cloud Run worker where deletion could occur before the execution/streaming fully settled. This was fixed in recent releases by relying on Cloud Run’s execution state (and deleting only after it’s no longer running).
A few concrete steps to get you unblocked:
- Upgrade to the latest Prefect and prefect-gcp
pip install -U prefect prefect-gcp
or with uv
uv pip install --upgrade prefect prefect-gcp
- Ensure your work pool is the Cloud Run V2 worker (recommended) rather than the legacy Cloud Run worker.
- Set a timeout that comfortably exceeds your flow’s expected run time (this is passed to Cloud Run and enforced natively):
work_pool:
name: your-cloud-run-pool
job_variables:
timeout: 3600
keep_job: false
- While debugging, keep keep_job: true so you can inspect the job and executions in GCP; switch back to false after upgrading/validation.
Could you share:
- Prefect and prefect-gcp versions
- Whether your pool is “Cloud Run V2” or legacy “Cloud Run”
- Your pool’s job_variables (timeout, cpu/memory, region)
- The specific error/log line that indicates premature termination
Quick version check:
python -c "import prefect, prefect_gcp; print('prefect:', prefect.__version__, 'prefect-gcp:', prefect_gcp.__version__)"
Helpful docs:
- Prefect GCP integration docs: prefect-gcp docs
- Prefect documentation: Prefect docs
If you’re already on the latest versions and still see this with keep_job=false, please share a run ID and a brief log snippet and I’ll dig in further.