https://prefect.io logo
Powered by
Title
t

Tim Wisniewski

03/19/2023, 2:38 PM
Hi folks, I'm trying to figure out the simplest and least expensive way to get my first Prefect flow into production. Am I correct in thinking I could just spin up a Digital Ocean $5/mo droplet, install 
prefect
on it, copy my workflow repo onto it, 
prefect login
to prefect cloud, and then have systemd run 
prefect agent start
? Or do I need to add a 'process block'?
r

redsquare

03/19/2023, 4:47 PM
Oracle cloud give you free vm's https://www.oracle.com/uk/cloud/free/
but yeah systemd your agent like you say
t

Tim Wisniewski

03/19/2023, 4:48 PM
Thanks. Currently trying to get it working on fly.io as well, though that's a bit more complex
Unfortunately, for both digital ocean and inside a container, I'm getting 
prefect.exceptions.MissingResult
errors 😞
r

redsquare

03/19/2023, 6:32 PM
on the back of what?
t

Tim Wisniewski

03/19/2023, 6:41 PM
The result was not persisted and is no longer available.
I'm assuming it's because I haven't configured a 'storage' block, but I'm struggling to wrap my head around why I'd need to in this simple setup. Here's my deployment config, if that helps (thanks so much for your time with this) https://gist.github.com/timwis/2e6aa51b803839a77229beea3342c777
r

redsquare

03/19/2023, 7:00 PM
are you trying to persist the result in the flow constructor
t

Tim Wisniewski

03/19/2023, 7:01 PM
Nope
Just added my flow code to that gist
r

redsquare

03/19/2023, 7:08 PM
perhaps the flow crashed and it is trying to persist results so it doesn't have to rerun
t

Tim Wisniewski

03/19/2023, 7:14 PM
oh, that's interesting. It probably did crash. But why does it have a problem persisting the results on the VM, but not on my laptop? 🤔
r

redsquare

03/19/2023, 7:15 PM
its just that you have not configured where to persist
so its defaulted to none, the error is just saying it hasnt saved anything
t

Tim Wisniewski

03/19/2023, 7:16 PM
Interesting... does that mean I need to define it in the 
storage
section of the deployment.yaml file? Or in the python code, when I call the flow? (like this https://docs.prefect.io/concepts/results/?h=persistence#result-storage-location)
r

redsquare

03/19/2023, 7:17 PM
the code is quite easy to follow https://github.com/PrefectHQ/prefect/blob/b153011b8bd5ac9ca6e59edb582e5018bae44a44/src/prefect/results.py#L363
The storage is where it downloads flows from
what you need is result_storage on the flow
see https://docs.prefect.io/concepts/results/#storage-of-results-in-prefect
t

Tim Wisniewski

03/19/2023, 7:22 PM
Thanks, but above that (here), it says:
The result storage location can be configured with the 
result_storage
option. The 
result_storage
option defaults to a null value, which infers storage from the context. Generally, this means that tasks will use the result storage configured on the flow unless otherwise specified. If there is no context to load the storage from and results must be persisted, results will be stored in the path specified by the 
PREFECT_LOCAL_STORAGE_PATH
setting (defaults to 
~/.prefect/storage
).
My take-away from that is that, if I don't specificly specify a result storage option, it will store results in 
~/.prefect/storage
r

redsquare

03/19/2023, 7:24 PM
so try persist_result:true and leave the storage
that is if you want to do this...first address whats crashing imo
t

Tim Wisniewski

03/19/2023, 7:26 PM
Sure, but I want to make sure the system can crash and record the error that's actually causing it to crash, rather than this error 😛
That seems to have helped! I got an error message from one of the APIs I'm hitting.
r

redsquare

03/19/2023, 7:27 PM
should be logs in prefect cloud on the flow run
t

Tim Wisniewski

03/19/2023, 7:27 PM
Thanks for your help
r

redsquare

03/19/2023, 7:27 PM
No worries, helps me to learn all of this too
#prefect-communityJoin Slack
Powered by