https://prefect.io logo
k

Kazushi Nagayama

09/03/2023, 5:55 AM
Feature request! Please accept Cloud Run timeouts bigger than 3600. Currently Prefect crashes with
Copy code
Worker could not generate job configuration: 1 validation error for PushPoolCloudRunWorkerJobConfiguration
timeout
  ensure this value is less than or equal to 3600 (type=value_error.number.not_le; limit_value=3600)
c

Chris White

09/03/2023, 7:25 AM
Hi Kazushi! Unfortunately that setting is out of our control and 3600 is the maximum that google allows: https://cloud.google.com/run/docs/configuring/request-timeout Your other request for subflows is something we can consider though 🙂
k

Kazushi Nagayama

09/03/2023, 7:28 AM
@Chris White my understanding is they started accepting timeouts longer than one hour as a preview feature for cloud run jobs! https://cloud.google.com/run/docs/configuring/task-timeout By default, each task runs for a maximum of 10 minutes: you can change this to a shorter time or a longer time up to up to 24 hours. Note that times greater than 1 hour are a Preview feature.
👀 1
Yeah subflows as different tasks would also be awesome :)
j

Jack P

09/04/2023, 5:10 PM
I have built a worker (for cloud run v2) and cloud run job v2 block that i have been using, just haven't had time to get worker ready for PR into Prefect-GCP (have one open for Cloud Run V2 Block that's tested/working, but only works with Agent so I think Prefect doesn't want to merge that without Worker too). Hopefully in the next month I can open PR after working more on worker. Here it is if you want to use, works for jobs up to 24 hours, but I need to clean Worker up a bit and test it a lot more, so use caution for now: https://github.com/japerry911/prefect-gcp/tree/skylord/new/worker_for_cloud_run_v2
🔥 2
k

Kazushi Nagayama

09/06/2023, 7:36 AM
oh nice! thanks for the update
j

Jack P

09/11/2023, 2:15 PM
Hey Kazushi, it's on this branch now https://github.com/japerry911/prefect-gcp/tree/skylord/new/cloud_run_v2_and_worker_v2-active Ended up rebuilding it, hoping to open PR in 3-4 weeks, just have some other priorities first. It's 80-90% built out, will work with runs, but won't
kill_infrastructure
isn't build yet in worker.
k

Kazushi Nagayama

09/13/2023, 3:58 AM
thanks for the update! for now i’ll resort to having an agent on compute engine vm. i’ll look forward to using this once it’s in prod!
🦜 1
j

Jack P

09/13/2023, 2:35 PM
I'll let you know when PR is open/will post in this thread in 3-4 weeks 😄 , thanks Kazushi! gcp !!