Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey all, I created a simple example showing how to deploy a Prefect agent to a docker on Google Cloud Compute Engine, using Github Actions.  Flow stored in Google Cloud Storage.  I had previously posted this when Prefect 2 was in beta, and just updated it for 2.0.1.  <https://github.com/BennyJW/prefect2-docker-gce>

Awesome work! is this line just for troubleshooting? <https://github.com/BennyJW/prefect2-docker-gce/blob/main/.github/workflows/deploy-to-gce.yaml#L47>

regarding: "Google will keep copying new docker images to the machine;" - I wonder if you could leverage image_pull_policy to solve it?

<@U02H1A95XDW>, yes you're right on those lines. It was just to confirm the connection to Prefect Cloud.

This looks awesome! I’ll try it out <@U02BG61CLJK>

<@U02H1A95XDW> I don't know about image_pull_policy.  One alternative is the docker prune command.  The problem is that when Google updates a Compute Engine with a new docker image, it doesn't delete the old images.  So eventually your Compute Engine will fill up with old images.  <https://stackoverflow.com/questions/69246222/remove-old-containers-for-google-container-os>

ahh you're right, since your tag is GITHUB_SHA :thinking_face:

still, it seems a bit drastic measure to delete VM every time. Perhaps you can run some maintenance flow that does docker image prune as shell command?