hey would anyone be able to help me setup up my prefect deployment against modal? I'm having a pretty confusing issue just trying to set where the working directory is. Seems pretty silly but I can't get prefect to direct to modal where the actual entrypoint of the code is

👋 hey @Ben Epstein - i responded to your issue, but since you're using

prefect.yaml

i'd guess something like this is what you want https://github.com/zzstoatzz/prefect-pack/blob/477484d91c77309a942b5ed9b7109ef48731f6df/prefect.yaml#L59-L61 which shouldn't require you to cd into the cloned repo note that my entrypoint is relative to the root of the repo being cloned

thanks! the issue with entrypoint needing to be relative to the root of the cloned directory is that this isn't where my flow is when I deploy it locally, which then causes

prefect deploy

to fail. My repo is setup like

repo
|_ project
|_ _ Makefile
|_ _ prefect.yaml
|_ _ flows
|_ _ _ my_flow.py

and so in my prefect.yaml i have my deployment as

entrypoint: flows/my_flow.py:main

but then when its cloned, it needs to be

project/flows/my_flow.py:main

> unless im always calling prefect deploy from the root of my directory this is generally what we recommend https://github.com/zzstoatzz/prefect-pack/tree/main/flows/nested_project so from the root of this repo i run

» prefect --no-prompt deploy --prefect-file flows/nested_project/prefect.yaml

» prefect deployment run 'some-flow/nested-project-flow'

» prefect worker start --pool 'local' # not required in your case

and since my

entrypoint

is still relative to the root of the repo (where I'm running the command from), this works as expected > i think a more scalable option could be to let

entrypoint

be a python module this might be possible, feel free to flesh out the idea in an issue!

but is there a way to work around this for me using

working_directory

now? I would definitely prefer not to restructure my entire project just to fit this parameter of where files exist for a prefect deployment. Is there no way currently to achieve what i'm trying to do?

sorry, i might be missing some detail, I'm not trying to suggest you need to change your entire setup, only that you should • specify

entrypoint

as a path that's relative to the root of the repo you're cloning • run

prefect deploy

from the root of that repo, using

--prefect-file

to point at the right yaml

i have multiple distinct projects in my repo, each in a folder 1-level down from the root, each with their own makefile. The "right" solution for my setup is to have the deploy command be in the respective makefile, but that would require not running it from the root of the repo, or adding like a

cd .. && prefect deploy

I could definitely create a makefile at the root just for prefect, it just feels a little silly to have it just for prefect. But i'll do one of those if there's no way to work around this with the

working_directory

step

gotcha the detail i was missing was that you want

make deploy-prefect

which necessitates running

prefect deploy

from the nested folder, and np - thanks for explaining!

hmm so i changed it to be at the root of the directory and it's somehow still throwing the same issue, now with the full relative path

prefect.exceptions.ScriptError: Script at 'copilot/src/flows/auto_respond.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')

even though the deploy does work from the root

sorry afk right now but that sounds excellent! it would be much appreciated 🙇

@Nate as promised 🙂 https://smiling-galliform-8d3.notion.site/Prefect-on-Modal-1912909a140e80d29ac7dd72558f3f3c

wow this is awesome! thanks so much 💙 are you open to us finding a home for this somewhere in the prefect domain?

Totally! Would love to get it on

Hey just joined and setting up prefect for the first time! @Ben Epstein I am following your guide now and will let you know if I hit any hiccups

Ben, not to pile on too many asks here, would you be willing to drop the modal guide in #CL09KTZPX? This is going to be so helpful for others to see an learn from. 🫶 You rock

Sure thing 🤝

Tutorial is great, but running into issues cloning a private repo. I know the credentials are right because I was using the block earlier to pull from my repo. Runner exception message is pretty vague, just "failed to clone"

Hm weird! I'm using a private repo for my real project and I'm using the same pattern. Just to sanity check, • You put this as the credential (not token) in the yaml:

credentials: '{{ prefect.blocks.github-credentials.prefect-modal }}'

• You put the PAT into the block in the perfect dashboard UI?

Rubber ducking always works so well -- it seems prefect init defaults to access_token but you clearly say to change that to credentials

Yes I found that so weird too haha. I'll add a more aggressive note in the doc!

Yep I see it on my end -- now my issue is with this pull YAML it says it is pulling:

# pull section allows you to provide instructions for cloning this project in remote locations
pull:
  - prefect.deployments.steps.git_clone:
      id: clone-step
      repository: "<https://github.com/MyOrg/my-repo.git>"
      branch: feat/prefect/init
      credentials: "{{ prefect.blocks.github-credentials.trey-gh-auth-token }}"
  - prefect.deployments.steps.run_shell_script:
      directory: "{{ clone-step.directory }}/services/prefect_cloud"
      script: make setup
  - prefect.deployments.steps.set_working_directory:
      directory: "{{ clone-step.directory }}/services/prefect_cloud"

FileNotFoundError: [Errno 2] No such file or directory: 'my-repo-feat/prefect/init/services/prefect_cloud' So parsing the branch as a directory path maybe? How is clone-step.directory set?

yes this got me stuck for a while. Make sure that the file exists in the branch that you are puilling from

the deployment step actually doesn't let you supply a name so if you supply a branch it will look for a file that doesn't exist

it's not a bug, it's just confusing behavior 🙂 https://github.com/PrefectHQ/prefect/issues/16998 what did you set your entrypoint to? it needs to be relative to

"{{ clone-step.directory }}/services/prefect_cloud"

yes that is the case -- the setup shell script isn't working because "{{ clone-step.directory }}/services/prefect_cloud" unfurls to my-repo-*{{branch}}*/services/prefect_cloud if you supply a branch

so modal pulls directly from github when runnign the code. So its doing a git clone. So for example, you have

repository: "<https://github.com/MyOrg/my-repo.git>"
branch: feat/prefect/init

it will

git clone <https://github.com/MyOrg/my-repo.git> && git checkout feat/prefect/init

and it will place you in the root of the repo

the path should not include the branch

nah it does that for main as well, it unfurls to

my-repo-main

Yeah I get what you are saying I think -- I'm just confused why the branch is in the path? if you

git clone

the repository, it isn't going to include that branch in the path

yea agreed, no idea why they do that

yeah... might take you up on the screen share

ooo hold on ill update the makefile, theres another way to install it

weird that you didn't hit this

its cuz i do

pip install --upgrade 'uv>=0.5.6,<0.6'

success! @Ben Epstein turns out that / is needed for the set_working_directory step -- not sure why

oh great thanks! ill add it back

i should have been more explicit, previously i think I was only trying to share the

git_clone

step, not the

run_shell_script

hack (that only works for a process pool running started from the same venv) to install some deps at runtime

ohh hm so how do you actually install your dependencies? i extrapolated that into the guide i wrote (and also am trying now haha). i have a

uv.lock

file that I need to get into whatever the python executable is

are you still using modal? i'd think you'd want some dockerfile that uv's all your deps into an image, or alternatively (more prone to weird outcomes) set

EXTRA_PIP_PACKAGES='pandas whatever git+{your repo}'

which would use uv (in the prefect base image) to install deps at runtime building an

image

and sticking that on your work pool is the normal pattern for containerized work pool types tho

is there an example of doing that with a

prefect.yaml

file? I assume it's in the

build

?

yep • docs • other example but you're totally free to leave image building out of prefect's purview and

docker build

/

docker push

in CI independently and reference the resulting

image

in your work pool

hm okay, i can try that. is there an example of

EXTRA_PIP_PACKAGES

in the docs? I see mentions of it but no actual usage

we could prob add it in the yaml section as well but yeah https://docs.prefect.io/v3/deploy/infrastructure-examples/docker#additional-configuration-with-deploy its a special env var, so in yaml itll look like

deployments:
    ...
    work_pool:
      name: your-work-pool
      job_variables:
        env:
          EXTRA_PIP_PACKAGES: pandas whatever

for context

awesome thanks!! I'll play around with this, feeling unblocked 🙏

ill update the guide when I figure this out