Hi does anyone know in Prefect 2 why there would b...
# prefect-community
Hi does anyone know in Prefect 2 why there would be a 30 second delay between the agent picking up a flow run and starting to execute it? Many thanks.
I have my flows stored in GitHub storage and I always assume this is the time it takes to clone everything and setup the infrastructure... But this is only an assumption and I would be happy to learn more about this!
Well I am seeing that it checks for runs every few seconds but the delay is after it picks up a run but before it starts executing. Seems it takes a long time to create a subprocess on the box the agent is on
In the subprocess it runs python -m prefect.engine
In general I also noticed there is a delay in running flows locally since we migrated to prefect 2
Probably linked to all the background stuff it needs to spin up in order to track the runs
Hi, I put in debug statements to the agent and it's the get_directory call on the storage which takes the time. Seems the subprocess spawned by the agent copies everything into a directory under the user's temp
Not sure why that is necessary really as I am using local storage so it does not need to copy from one local location to another
permissions might not be the same for the directories locally, since where your code and execution is stored is presumably owned not by the prefect account
you might have read permissions for the user, but not execute permissions
Actually I found a way round the copy. I created a Process block with the working directory the same as the one I use for the deployment. No copy is therefore required and it works really nicely.
🎉 1