Moving this post here since this seems more releva...
# prefect-cloud
n
Moving this post here since this seems more relevant: Has anyone been struggling with getting the agent to now stall on downloading code before running a deployment? I'm downloading a good bit of code, but when I run a manual s3 download on command line, it completes quickly. Also, I can run this code snippet:
Copy code
from prefect.filesystems import S3
s3_block = S3.load("flow-code")
s3_block.get_directory(from_path="/", local_path=".")
and it also downloads in a couple of seconds however, when I run the prefect-agent, it stalls at the download. It stalls whether I use an S3 storage block or if I try to run it locally. Frustrating because everything was working fine until it just... wasn't anymore. Has anyone else seen this? One difference I see in the code snippet to what is happening in prefect-agent code is that it is being run async, but I didn't think that should be an issue
c
Hey Nicholas - I've seen this happen before when the s3fs library and fsspec get out of sync / have incompatible versions, so I suggest checking what versions you have locally and what versions are running in your deployment's execution environment
n
hmmm, I'm running the agent on my laptop right now. I can also run this code locally and it works fine
Copy code
from prefect.filesystems import S3
s3_block = S3.load("amplify-flow-code")
s3_block.get_directory(from_path="", local_path=".")
c
and you're not using virtual environments or docker or anything like that?
n
I’m running the agent in the same conda env im running that code snippet in
Interestingly, it also hangs there when I try to run it with local code
c
innnnteresting, the plot thickens
i think you should open an issue for this on github
n
doing that now. will send link in a minute
šŸ™ 1
so I made a simple flow example, and it works fine. In my more complex one, I am trying to include a django application in my code so I can access the django models and db from the task. Something about copying over the entire app is causing it to freeze? Trying to figure out how to reproduce it for the ticket
r
I have similar issues with big flow code. In my case, I have an EC2 instance on which I have the code, I deploy it without specifying a storage (hence using local storage?) and then I run the agent on that EC2 instance. However, download also takes a long time (although it is local, so what takes long time? does it copy code over locally?). Unfortunately, there are even errors
No such file or directory
when downloading the code, so that the deployment currently does not run successfully šŸ˜•
Could it be related to
micromamba
vs
conda
?