Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
m
MasatoShima
08/13/2022, 1:38 PMHi.
I’m trying Prefect Orion.
I tried to run deployment.
However, the flow status stays ‘pending’ and is not executed.
Does anyone have any idea what the reason might be?
I would be happy to know.
The following steps were executed.
--------
# Prefect orion start.
% prefect orion start
___ ___ ___ ___ ___ ___ _____ ___ ___ ___ ___ _ _
| _ \ _ \ __| __| __/ __|_ _| / _ \| _ \_ _/ _ \| \| |
| _/ / _|| _|| _| (__ | | | (_) | /| | (_) | .` |
|_| |_|_\___|_| |___\___| |_| \___/|_|_\___\___/|_|\_|
Configure Prefect to communicate with the server with:
prefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
View the API reference documentation at <http://127.0.0.1:4200/docs>
Check out the dashboard at <http://127.0.0.1:4200>
INFO: Started server process [26251]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on <http://127.0.0.1:4200> (Press CTRL+C to quit)
...
--------
# Build deployment.
% poetry run prefect deployment build ./workflows/flow_hello.py:main_flow -n flow_hello -t test
Found flow 'flow_hello'
Deployment YAML created at '/********/********/********/********/main_flow-deployment.yaml'.
--------
# Set 'PREFECT_API_URL'.
% prefect config set PREFECT_API_URL=<http://127.0.0.1:4200/api>
Set 'PREFECT_API_URL' to '<http://127.0.0.1:4200/api>'.
Updated profile 'default'.
--------
# Apply deployment.
% prefect deployment apply main_flow-deployment.yaml
Successfully loaded 'flow_hello'
Deployment 'flow_hello/flow_hello' successfully created with id '04a098ba-b53b-4216-98a2-9c96b059104f'.
--------
# Start Prefect Orion Agent.
% prefect agent start -t test
Created work queue 'Agent queue test'
Starting agent connected to <http://127.0.0.1:4200/api>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue 'Agent queue test'...
--------
# Run deployment.
% prefect deployment run flow_hello/flow_hello
Created flow run 'cerise-adder' (fc7e440d-05cb-4228-83a0-de7f70015531)
--------
# After execute command 'prefect deployment run flow_hello/flow_hello', output folling logs by Prefect Orion Agent.
22:27:59.810 | INFO | prefect.agent - Submitting flow run 'fc7e440d-05cb-4228-83a0-de7f70015531'
22:27:59.877 | INFO | prefect.infrastructure.process - Opening process 'cerise-adder'...
22:27:59.884 | INFO | prefect.agent - Completed submission of flow run 'fc7e440d-05cb-4228-83a0-de7f70015531'
--------t
Taylor Curran
08/13/2022, 1:42 PMHi Masato, you have each one running in a separate terminal right?
m
MasatoShima
08/13/2022, 1:43 PMYes.
t
Taylor Curran
08/13/2022, 1:45 PMStrange. Looks like you build your deployment with tag
testSomehow the agent isn’t picking up on the flowrun
Wonder if it might be poetry somehow?
m
MasatoShima
08/13/2022, 1:50 PMStrange. Looks like you build your deployment with tag test and started an agent using the same tag so they should work.I think so too. I created a new queue and tried it. However, it didn’t work…
prefect agent start 7a9b41ee-1e94-4f9d-b86f-865289ed0433t
Taylor Curran
08/13/2022, 1:52 PMI’m trying to find any discourse discussing poetry and coming up short 😕 (though tbh i could have looked harder so don’t let that stop you from further digging 😅) I wonder if its worth building the same test with a venv or conda environment? I’ve just heard people struggling with poetry in the past.
I’m sure poetry is doable, but just might be useful to isolate that as the problem.
m
MasatoShima
08/13/2022, 2:06 PMI tried with venv.
However I got the same result...
t
Taylor Curran
08/13/2022, 2:07 PMHave you tried running the flow from the UI?
Going to Deployments -> test_flow/test_deployment -> clicking the run button on top right
m
MasatoShima
08/13/2022, 2:13 PMYes.
Tried both UI and CLI.
t
Taylor Curran
08/13/2022, 2:18 PMMaybe creating a work-queue and using the work-queue id in the
agent start commandDouble check that the terminal where you are running the agent, when you run
prefect deployment lsAlso might be an issue coming from profile somehow. Sometimes I reset that file by just deleting it
~/.prefect/profiles.tomlm
MasatoShima
08/13/2022, 2:42 PMMaybe creating a work-queue and using the work-queue id in theinstead of a tag?agent start command
…
Sometimes I reset that file by just deleting itI got the same result....~/.prefect/profiles.toml
Double check that the terminal where you are running the agent, when you runI tried it. There seems to be no problem.you see your test deployment listed. Same with work-queue.prefect deployment ls
I am stumped, but ultimately looks like things are working just your agent and work-queue aren’t talking properly for some reason. Sorry I can’t be of more help.Don’t worry. I thank you for your help.
a
Anna Geller
08/13/2022, 3:03 PMWe've seen several people having import issue using venv, can you try with Conda instead?
Alternatively destroy your current venv and create a new one from scratch, activate it before starting the agent, same for the terminal from which you create deployment
m
MasatoShima
08/13/2022, 4:10 PMThanks. I will try it.
I noticed that it seems to be slower than it should be until the flow is executed.
After running the deployment run, it runs 30 minutes ~ 1 hour later, and the status changes to ‘Completed’.
a
Anna Geller
08/13/2022, 6:39 PMyup, changing to conda should fix it for you
m
MasatoShima
08/16/2022, 6:15 PMHi.
Unfortunately, I was not able to solve this problem using conda.
I finally solved this problem by using this Container Image
It’s a little unclear, but I’ll try to work this way.
https://hub.docker.com/r/prefecthq/prefect
Thank you very much for all your advice.
😛anda-dancing: 1
t
Taylor Curran
08/16/2022, 6:17 PMSo glad you got it working and thank you so much for following up Masato!
:thank-you: 1