<@ULVA73B9P> When working with prefect helm locall...
# ask-marvin
d
@Marvin When working with prefect helm locally, what is the helm configuration in values.yaml for mounting local code
m
The documentation doesn't provide explicit details on mounting local code using Helm for Prefect, but you can generally achieve this by modifying the
values.yaml
file. You would typically set up a volume mount in the
values.yaml
to point to your local code directory. This involves specifying the volume and the mount path in the values file. Here's a rough example of how you might configure the `values.yaml`:
Copy code
yaml
worker:
  volumeMounts:
    - name: local-code
      mountPath: /app/local-code
  volumes:
    - name: local-code
      hostPath:
        path: /path/to/your/local/code
This example assumes you're using Kubernetes'
hostPath
to mount a local directory into the container at
/app/local-code
. Adjust the paths according to your local setup. For more detailed instructions and examples, you might want to refer to the Prefect Kubernetes deployment documentation.
d
@Marvin the files are not inside the worker
m
The documentation does not provide explicit details on configuring
values.yaml
for mounting local files directly into a Prefect worker when using Helm. However, a common approach involves using Kubernetes
PersistentVolume
and
PersistentVolumeClaim
to manage local file storage. Here's a general idea of how you might configure this: 1. Define a PersistentVolume in your Kubernetes cluster that points to your local files. 2. Create a PersistentVolumeClaim that requests storage from the PersistentVolume. 3. Modify the values.yaml to include these claims in your Helm chart. Here's a rough example for your `values.yaml`:
Copy code
yaml
worker:
  volumeMounts:
    - name: local-files
      mountPath: /app/local-files
  volumes:
    - name: local-files
      persistentVolumeClaim:
        claimName: your-pvc-name
Ensure that your Kubernetes cluster is set up to recognize and bind the PersistentVolume to the PersistentVolumeClaim. You might need to manually create these resources using
kubectl
. For more precise guidance, you may want to explore Kubernetes documentation on Persistent Volumes and Prefect's Kubernetes deployment documentation.
n
hi @datamongus - the server doesn't need access to your code deployments, which are stored on the server, will have some knowledge of where code lives via the pull step so that workers (which run client side) can
pull
down your code at runtime, but the server never needs your actual code
its possible you're not talking about your flow code when you mention "mounting local code" but feel free to clarify which code you mean
d
Yes I am referring to the flow code. In a local environment I would rather not pull the code into kubernetes everytime there is a code change. I prefer that my team is able to make changes to their flows and simply start a new run or deployment from the UI to test said changes. LMK if that makes sense.
n
well, when you change your code, where do you want to push it?
in my experience, github/gitlab is the answer (regardless if you're using prefect or not) which is why we have the
git_clone
pull step (executed by the worker clientside when its time to run a flow) it sounds like you're suggesting that you'd push your code to a local file system that happens to be mounted to the server?
d
I think I see the confusion. I’m ok with the build process and building the changes into a Dockerfile. I’m strictly referring to local development where I am working on my flow code, and want to test my changes in my local Kubernetes environment. For example, in the docker-compose file for local development you would mount your “flows” or “src” directory to a path within the worker and server. So when you deploy the flow the worker within the docker environment would have access to the code.
n
ah I guess I erroneously assumed you meant helm chart for prefect server, not the prefect worker i can't say I'm so sure what the best pattern would be there. I would probably tend towards building my local code into images that I then use in my local k8s setup not to beat a dead horse, but again in general I prefer github because that's where I push my code anyways and that way the mechanics of retrieving code are not different between dev and prod (im just changing the branch im pulling from, for example)
👍 1
d
This makes sense, thank you
👍 1
a
@datamongus if you want to iterate in your local k8s cluster, you create a work pool that has a volume mount declared in the base job template, similar to what you would do with Docker Compose. You might need to mount into the same spot that you put your code when your built your custom image. That's mostly theoretical, I haven't tried it myself! I guess the TL;DR is that it might be possible, but it'd be tough. I like your thought process, and we might be able to find a way to make that experience more ergonomic.
d
Thank you
I’m working on this now, I’ll share my findings.
👍 1