Feature request! Please accept Cloud Run timeouts ...
# prefect-cloud
k
Feature request! Please accept Cloud Run timeouts bigger than 3600. Currently Prefect crashes with
Copy code
Worker could not generate job configuration: 1 validation error for PushPoolCloudRunWorkerJobConfiguration
timeout
  ensure this value is less than or equal to 3600 (type=value_error.number.not_le; limit_value=3600)
c
Hi Kazushi! Unfortunately that setting is out of our control and 3600 is the maximum that google allows: https://cloud.google.com/run/docs/configuring/request-timeout Your other request for subflows is something we can consider though 🙂
k
@Chris White my understanding is they started accepting timeouts longer than one hour as a preview feature for cloud run jobs! https://cloud.google.com/run/docs/configuring/task-timeout By default, each task runs for a maximum of 10 minutes: you can change this to a shorter time or a longer time up to up to 24 hours. Note that times greater than 1 hour are a Preview feature.
👀 1
Yeah subflows as different tasks would also be awesome :)
j
I have built a worker (for cloud run v2) and cloud run job v2 block that i have been using, just haven't had time to get worker ready for PR into Prefect-GCP (have one open for Cloud Run V2 Block that's tested/working, but only works with Agent so I think Prefect doesn't want to merge that without Worker too). Hopefully in the next month I can open PR after working more on worker. Here it is if you want to use, works for jobs up to 24 hours, but I need to clean Worker up a bit and test it a lot more, so use caution for now: https://github.com/japerry911/prefect-gcp/tree/skylord/new/worker_for_cloud_run_v2
🔥 2
k
oh nice! thanks for the update
j
Hey Kazushi, it's on this branch now https://github.com/japerry911/prefect-gcp/tree/skylord/new/cloud_run_v2_and_worker_v2-active Ended up rebuilding it, hoping to open PR in 3-4 weeks, just have some other priorities first. It's 80-90% built out, will work with runs, but won't
kill_infrastructure
isn't build yet in worker.
k
thanks for the update! for now i’ll resort to having an agent on compute engine vm. i’ll look forward to using this once it’s in prod!
🦜 1
j
I'll let you know when PR is open/will post in this thread in 3-4 weeks 😄 , thanks Kazushi! gcp !!
There is an issue in the Google API Client Library, where a parameter that exists is not available for use. So I posted issue to Google API Python Client GitHub, and will be able to finish this worker when that's done. Everything but tests/cancelling is done.
j
Any news on this @Jack P? 🥺
j
@Johan sh it's in UI 🙂 catjam
❤️ 1
and in prefect GCP repo now 🦜
❤️ 1
j
Oh wow ! so great, thank you so much 🥰
🦜 1
🫡 1