https://prefect.io logo
Powered by
Title
c

Chris McClellan

03/29/2023, 12:23 AM
OK, I'm trying to get Prefect Cloud v2 working .... and learning as I go .... I have a script that is working perfectly (finally), now created a deployment with an "every 30 mins schedule" .... but does that mean that every workflow in the workspace runs on that schedule?
r

redsquare

03/29/2023, 5:27 AM
No just that flow you deployed
c

Chris McClellan

03/29/2023, 5:27 AM
I can't get the flow linked to the deployment
r

redsquare

03/29/2023, 6:06 AM
What does your deployment look like
c

Chris McClellan

03/29/2023, 6:08 AM
in what regard ? I'm using Cloud v2. I defined the deployment, added a schedule of 0,30 * * * *
it's enabled, and creates the runs ... but nothing actually runes
runs 😛
r

redsquare

03/29/2023, 6:11 AM
Have you got an agent running to pick up the flow run?
c

Chris McClellan

03/29/2023, 6:11 AM
yes
running via supervisord
well, I THINK it's running ... maybe not 😞
r

redsquare

03/29/2023, 6:12 AM
check the agent logs
it should show in the cloud ui as being healthy if connected
c

Chris McClellan

03/29/2023, 6:14 AM
where is that in v2 ?
hmm m... even with Cloud do I need to do :
prefect deployment build
I'm always confused what is meant for when you're using Cloud or not
r

redsquare

03/29/2023, 6:19 AM
in v2 there is a work pools menu on the left hand nav
inside the pool there is work queues showing your agent(s)
e.g
c

Chris McClellan

03/29/2023, 6:21 AM
UNHEALTHY 😞
r

redsquare

03/29/2023, 6:21 AM
you need to do build and apply
on your deployment
if its unhealthy - are you sure the api key and url env vars are set correctly
check the logs
c

Chris McClellan

03/29/2023, 6:22 AM
I remember pasting an API this morning 🙂
Basically I created a new VM this morning, installed prefect and did all the setup
copied over my scripts and converted to v2
if I run manually they work fine, I just can't schedule them
on my old (v1) VM that I was using with Prefect Cloud v1 I had to run supervisord to make sure that the agent was alive all the time
r

redsquare

03/29/2023, 6:25 AM
the agent needs to be healthy to poll for flows on the schedule
c

Chris McClellan

03/29/2023, 6:25 AM
so I copied that over to the new VM (the config file after installing supervisord) but there's still something that I'm missing (in my brain, in the config, somewhere)
r

redsquare

03/29/2023, 6:25 AM
agent needs PREFECT_API_KEY and PREFECT_API_URL
c

Chris McClellan

03/29/2023, 6:26 AM
hmm ... I just started the agent on the CLI and it's running all the flows that hadn't started yet
agent is now showing as Healthy in the Work Queues tab
I just need to get the agent working all the time
I think that might be the missing piece
r

redsquare

03/29/2023, 6:27 AM
systemd
c

Chris McClellan

03/29/2023, 6:27 AM
not supervisord ?
r

redsquare

03/29/2023, 6:28 AM
it should work - not tried it
the supervisord logs should say what is happening if it cant keep the agent healthy
how to systemd it https://discourse.prefect.io/t/how-to-run-a-prefect-2-agent-as-a-systemd-service-on-linux/1450
c

Chris McClellan

03/29/2023, 6:41 AM
ummm ... that looked like it worked 🙂
agent is reporting healthy now
r

redsquare

03/29/2023, 6:42 AM
which bit 😁
c

Chris McClellan

03/29/2023, 6:44 AM
no, I spoke too soon 😞
in Work Pools > Work Queues the agent is healthy
in Work Pools > Runs the deployment runs
but the flow still isn't linked to the deployment
is this a CLI thing I need, or something in the actual code ?
I'm missing the yaml file I think
r

redsquare

03/29/2023, 6:51 AM
you build the yaml file then apply to deploy
c

Chris McClellan

03/29/2023, 6:51 AM
I still need to do that with Cloud ?
r

redsquare

03/29/2023, 6:51 AM
yes
you do using python to deploy
cli is --apply
https://docs.prefect.io/concepts/deployments/#deployment-build-options
c

Chris McClellan

03/29/2023, 6:58 AM
can I have 2 flows in a deployment ?
I can't even get the yaml file to be created, or it's not where I think it should be 😞
r

redsquare

03/29/2023, 7:10 AM
you can bundle a second flow and call it as a subflow
use --output to specify the folder for the yaml
c

Chris McClellan

03/29/2023, 7:13 AM
OK, so 1 flow per deployment is fine, I just can't get even 1 deployment to work
r

redsquare

03/29/2023, 7:19 AM
if you specify the output path it should create the yaml, assume you have specified storage/infrastructure etc
c

Chris McClellan

03/29/2023, 7:40 AM
I'm running on a linode VM
so it's just like a normal machine
r

redsquare

03/29/2023, 7:42 AM
so you want it running as a sub process on the agent
c

Chris McClellan

03/29/2023, 7:43 AM
I don't know what the right terminology is - I just want my 2 scripts running every 30 mins like they did on Prefect Cloud v1
it just feels like there's far too many hoops to jump through to even get one flow running on a schedule 😞
and I know it's just me ... surely it can't be the product
r

redsquare

03/29/2023, 7:44 AM
yeah so youll need a process infrastructure block to it knows how you want the flow to be ran once the agent collects it
and youll need a storage block so it can download the flow files
c

Chris McClellan

03/29/2023, 7:45 AM
so I need to write the code, then make it a flow then create a deployment then assign the flow to a deployment (I still can't do that), but now I need a storage block and ... andd ... and.... it's just too much 😞
the deployment works, it just doesn't know what to run
r

redsquare

03/29/2023, 7:47 AM
it's all pretty logical, it has to upload the files somewhere for them to be downloaded and ran
c

Chris McClellan

03/29/2023, 7:47 AM
but they run fine on the VM
can't they just run "in situ"
r

redsquare

03/29/2023, 7:48 AM
then a localstorage block
c

Chris McClellan

03/29/2023, 7:48 AM
why a block at all?
I'm reading all that doco about deployments and I haven't seen any mention of a block yet
but rewinding back to the start .... I have a VM that has 2 python scripts. Both scripts have to run every 30 mins (in cron terms 0,30 **)
that's it, that's all I need
r

redsquare

03/29/2023, 7:50 AM
i thick local storage is the default
c

Chris McClellan

03/29/2023, 7:50 AM
Prefect 1 had a great gui and was very simple to configure and use - v2 is a nightmare
r

redsquare

03/29/2023, 7:50 AM
as long as its in the default prefect path
p.s I dont work for Prefect:)
and I didnt use v1
c

Chris McClellan

03/29/2023, 7:51 AM
yeah, I know 😞 .... I've had meetings with them as well - hence why I'm here because they stuffed my v1 setup this morning 😞
r

redsquare

03/29/2023, 7:51 AM
https://docs.prefect.io/concepts/storage/
c

Chris McClellan

03/29/2023, 7:51 AM
so this MUST be fixed today and it still doesn't work
I can run the scripts manually, it works fine and shows the results in the GUI
surely the "scheduling" side should be a breeze to configure and use - that's the whole point right ?
r

redsquare

03/29/2023, 7:54 AM
it is, just a cron or rrule
c

Chris McClellan

03/29/2023, 7:54 AM
I mean ... get a script to run on the cron.
my deployment is working fine, just not actually doing anything
r

redsquare

03/29/2023, 7:54 AM
so if your flows in the vm are in PREFECT_LOCAL_STORAGE_PATH it should find them and invoke them
so that is ~/.prefect/storage
by default
c

Chris McClellan

03/29/2023, 7:59 AM
I don't even have a storage folder in .prefect
and no PREFECT_LOCAL_STORAGE_PATH defined
in .prefect there is only a profiles.toml
r

redsquare

03/29/2023, 8:02 AM
did you install in a venv
c

Chris McClellan

03/29/2023, 8:02 AM
no, just a brand new VM
only for this purpose
r

redsquare

03/29/2023, 8:07 AM
what instructions did you follow
c

Chris McClellan

03/29/2023, 8:08 AM
LOL ... the instructions from Prefect employees and the website
pip install prefect, etc....
convert v1 to v2 doing this, etc..
"should be easy" .. just not so easy
I think I'm close, but at the same time I don't know how big the gap is - the script runs, you can see progress in the GUI
it just doesn't run every 30 mins and only runs when I manually run the script or the agent
actually ... if the agent is healthy why doesn't it work ?
r

redsquare

03/29/2023, 8:11 AM
the agent logs should give an indication if its picking up a flow run and then not finding the flow locally
c

Chris McClellan

03/29/2023, 8:11 AM
where are the agent logs ?
r

redsquare

03/29/2023, 8:18 AM
Logs to console by default - you can specify create a logging config to write it to disk
but lets start the agent in interactive mode and see what is happening
stop the systemd prefect service and start the agent from the command line
c

Chris McClellan

03/29/2023, 8:20 AM
like this ? systemctl stop prefect-agent.service
r

redsquare

03/29/2023, 8:23 AM
systemctl stop prefect-agent
c

Chris McClellan

03/29/2023, 8:23 AM
ok
r

redsquare

03/29/2023, 8:25 AM
then export PREFECT_LOGGING_LEVEL="DEBUG"
then start the agent interactively prefect agent start
c

Chris McClellan

03/29/2023, 8:26 AM
I have to do prefect agent start -p default-agent-pool
to provide the work queue
r

redsquare

03/29/2023, 8:26 AM
yeah work pool
c

Chris McClellan

03/29/2023, 8:27 AM
image.png
just constantly pinging
r

redsquare

03/29/2023, 8:27 AM
right so you have not set your cloud url
because it should tell you the workspace your connected to
export PREFECT_API_URL="https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]"
export PREFECT_API_KEY="[YOUR-API-KEY]"
c

Chris McClellan

03/29/2023, 8:29 AM
it does, I just didn't capture that 😉
r

redsquare

03/29/2023, 8:29 AM
ha ok
so have you got a deployment scheduled
c

Chris McClellan

03/29/2023, 8:30 AM
image.png
image.png
r

redsquare

03/29/2023, 8:32 AM
so 28 minutes
c

Chris McClellan

03/29/2023, 8:33 AM
hang on, it jsut caught this ...
image.png
but the flows still didn't run
r

redsquare

03/29/2023, 8:35 AM
says they completed
what does the flow do
c

Chris McClellan

03/29/2023, 8:36 AM
reads an API and writes the results
so I can check that no results have been written
image.png
those times match the most recent data, which is when I ran the scripts manually
times are Sydney time, so 5 hrs ago
r

redsquare

03/29/2023, 8:46 AM
have you got persist_result on the flow
c

Chris McClellan

03/29/2023, 8:47 AM
probably not, I don't know what that means at all
r

redsquare

03/29/2023, 8:49 AM
https://docs.prefect.io/concepts/results/#persisting-results
c

Chris McClellan

03/29/2023, 8:49 AM
image.png
I just ran the script manually at the command line - worked perfectly
r

redsquare

03/29/2023, 8:49 AM
it only persists if you tell it to - or set the env var to always persist
c

Chris McClellan

03/29/2023, 8:50 AM
do I need that though ?
r

redsquare

03/29/2023, 8:51 AM
dunno - you mentioned "no results have been written"
I assumed you meant flow result storage
c

Chris McClellan

03/29/2023, 8:51 AM
sorry, the data that is produced in the script
the script reads an API and writes data to a database
and I can check what has been written to the database
there is only something written when I run the script manually
even though the deployment is running every 30 mins
r

redsquare

03/29/2023, 8:58 AM
is that the flow that logs out Hi From Prefect
I would add some more logs to the flow before+after trying to write to the db to see if it gets to those points during the scheduled run
the scheduled ones seem to run for a second versus the 30+ of your manual ones
c

Chris McClellan

03/29/2023, 9:17 AM
image.png
that's the manual run, I just can't get the same thing on the schedule
r

redsquare

03/29/2023, 9:18 AM
yeah so log some stuff out so we can identify why as the agent log just says its completed
c

Chris McClellan

03/29/2023, 9:19 AM
hmmm the Hi is something I was trying earlier
r

redsquare

03/29/2023, 9:19 AM
seems to fall out quickly
c

Chris McClellan

03/29/2023, 9:19 AM
image.png
r

redsquare

03/29/2023, 9:20 AM
all the flow is doing is logging hi
c

Chris McClellan

03/29/2023, 9:20 AM
yeah, but at least it works
maybe that code should go in my script ?
r

redsquare

03/29/2023, 9:20 AM
where is the api call etc
c

Chris McClellan

03/29/2023, 9:21 AM
the Hi is just some dummy code I found today
I didn't even realise it was running properly
r

redsquare

03/29/2023, 9:21 AM
ok - so yeah - slap the actual work in the flow and you are good
c

Chris McClellan

03/29/2023, 9:31 AM
OK, that looked like it worked
at least they ran on scheduled
OK, THANK YOU SO MUCH for your help and time
I'll let it run overnight and see that it's still stable in the morning
r

redsquare

03/29/2023, 9:33 AM
cool, fingers crossed
c

Chris McClellan

03/29/2023, 9:40 PM
AGAIN ... thank you so much for your help yesterday, it's now working perfectly every 30 mins
r

redsquare

03/29/2023, 10:46 PM
hey @Chris McClellan, glad we sorted 👍
✅ 1
#prefect-cloudJoin Slack
Powered by