Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
f
Faheem Khan
08/29/2022, 7:50 AMHi everyone, am I missing something in the latest upgraded prefect from 2.0.4 to 2.2.0. I started agent with a random queue name i.e. q1 and added "--work-queue q1" argument to prefect deployment. now when I start my flow it freezes after sometime. the same code was working fine with prefect 2.0.4. I am using Dask, docker containers and minio storage. Cheers
✅ 1
a
Anna Geller
08/29/2022, 8:50 AMThe queue name set on an agent must match the name set on a deployment. Perhaps you can start from scratch in a new environment and ensure that they have a matching queue name and that both use the latest version?
👍 1
f
Faheem Khan
08/29/2022, 9:34 AMI can see my work q picked up by the deployment. Also if u check the flow run image Work Q is not picked by the flow, is it normal?
a
Anna Geller
08/29/2022, 1:55 PMcheck out this page - it will clarify all the confusion about work queues, agents and deployments https://discourse.prefect.io/t/prefect-deployments-faq/1467
f
Faheem Khan
08/29/2022, 10:54 PMThanks @Anna Geller for the response but I have been running my flows with prefect 2.0.4 successfully, with the new upgrade to prefect 2.2.0 I am facing issues. the flow run for while, it processes some files and then it freezes prefect versions are all same in agent and server.
a
Anna Geller
08/29/2022, 11:54 PMnot sure what went wrong there, it might be easier to upgrade and restart in a new environment
f
Faheem Khan
08/30/2022, 12:14 AMsomething definitely going with prefect 2.2.0, I even ran my flow scripts within prefect server directly from python same result. I rolled back to 2.0.4 by using docker prefect 2.0.4 image and everything works fine.
🙌 1
@Anna Geller the same code with prefect 2.2.0 runs fine on windows(windows), but freezes on linux.
Update: it happened with windows as well, after processing 20k files.😒
a
Anna Geller
08/30/2022, 11:09 AMWhat does it mean when you say freezes? If your agent cannot handle the flow runs with 20k files, you may consider deploying agent to a bigger machine or cluster like Kubernetes
If your execution layer freezes due to the load, you may consider signing up for Cloud (free tier) which we monitor for performance app.prefect.cloud
f
Faheem Khan
08/31/2022, 3:03 AMAgent submit all the the flow runs, by freezes I mean I can see pending tasks in the UI, completed task but no running tasks, also no tasks failed. and the flow container still running as expected. the same process works fine for prefect2.0.4, I am using same postgress server for prefect and for my script to store file names with max connections 150.
a
Anna Geller
08/31/2022, 4:59 AMThere are many potential reasons for pending state - checking agent logs might be a good first step for troubleshooting