https://prefect.io logo
Join Slack
Powered by
hey <@ULXMV9SD7> has your team ever seen prefect a...
# prefect-cloud
n
hey @Jenny has your team ever seen prefect agents running locally on a windows laptop stalling at the download code from storage phase? It is a local process, so trying to copy code from a local directory, but it never completes. The same code works on my ubuntu laptop. We tried running the agent in a docker container as well, but still stalls at the same step
j
Let me check with the team on this one!
Double checking - do you have the right filesystem package dependencies installed: https://docs.prefect.io/latest/concepts/filesystems/#filesystem-package-dependencies
n
if they are running the prefect agent locally on a windows laptop, would they need any of these installed?
only place windows is referenced is smb, but when the agent is running locally, the flow isn't ubuntu basee
*based
j
I guess I should ask my question a different way - when you say "download code from storage" what storage are you using? Can you share any error messages?
a
jumping in here as I'm facing this issue: just local file storage. No error message - the flow just never starts - it always stays in pending
j
Hmmm... One of our engineers kindly spun up a windows VM to test and they can't reproduce.
Double checking here @Ameya are you on 2.10.11? Want to make sure we're testing the right thing.
a
I was on 2.10.5, but I just upgraded and same issue
I see the submitted run in the UI, my local agent is "checking for scheduled flow runs...", I see 
INFO    | Flow run 'adept-panther' - Downloading flow code from storage at None
, but then the flow never moves past this step
flow run id 
8659bc10-46a5-4301-ac08-18e04ca32aa3
if helpful
o actually it looks like it finally ran! but it was 8 minutes of downloading flow code
any tips for getting that to be faster?
n
Compare your local files to the .perfect ignore
My best guess is a big csv file is sneaking in somehow
a
is there anyway to check what prefect is actually downloading? not seeing any big files that aren't in prefect ignore
@Jenny - do you know if there is a way to check what prefect is downloading?
j
Hey @Ameya! (Copying this from an earlier conversation) Perhaps have your flow print the current directory name then sleep forever on start then go there and look at the files? It's also just going to be a copy of "some/location". Do you have large files there? We recursively copy directories, do you have a lot of files in nested directories? Do you have a 
.git
directory there? What's excluded by your 
.prefectignore
? There's an older issue here that could be useful for debugging your flow as it's going through the steps you are asking about: https://github.com/PrefectHQ/prefect/issues/8178
Did this help you or are you still seeing the issue?
a
still seeing the issue - I have most things excluded by the .prefectignore, but trying to see what is slipping in
👍 1
ok I found what is being included, and it looks like the .prefectignore file is not being respected. It looks like the issue was opened here but never resolved: https://github.com/PrefectHQ/prefect/issues/8195. @Jenny - can someone take a look at this?
j
Ah thank you for the detective work. I've re-opened that issue. If you're creating new deployments I do recommend the new projects pattern as I find it quite a bit simpler but I'll flag this to the team working on deployments.
n
@Jenny is there a timeline for when projects will be not a beta feature?
Also, this seems like an issue with the prefect agent, not the deployment itself. For our deployment using a LocalProcess, it looks like this is the line of code that copies over the directory: https://github.com/PrefectHQ/prefect/blob/929234e85ea80dc4fe6ac3662e4955db75f925d4/src/prefect/filesystems.py#L124 Which doesn't seem to account for the prefectignore
oh wow, @Ameya, he actually points this out here: https://github.com/PrefectHQ/prefect/issues/8195#issuecomment-1387418643 so the local copy process doesn't respect the prefectignore right now, which is why that is happening
j
Thanks @Nicholas Torba - I've asked the team to look into this issue. I don't have an exact date for when the projects/new deployments will be out of beta but we are actively working on them to make the UX as smooth as possible.
This should resolve the .prefectignore issue: https://github.com/PrefectHQ/prefect/pull/9863
n
awesome!
a
Looks like it’s been merged, which is great! Does that mean it’s immediately available or do we have to wait until the next release?
j
It was included in yesterday’s release: https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md
❤️ 1
a
things are working now for me - thanks for all the help here!
j
Great! Thanks for confirming.
Open in Slack
PreviousNext
Powered by