Hello everyone! I have a question about having an ...
# prefect-server
j
Hello everyone! I have a question about having an option of running the same workflow both with the prefect server and without it. For example, when we just want to evaluate the correctness of the workflow and write some unit tests for it, we run it in a console without the server, using
LocalResult
. But then we'd like to register the same workflow on the server and run it there on a large scale, using
PrefectResult
and cache to the database. We're struggling to get the
LocalResult
and
PrefectResult
behave in the same way, so we could just switch between one and the other. Does this make sense at all - to have a switch between offline and server run? Or should one result class handle all cases, perhaps a custom one?
k
Hey @Jovan Visnjic, The main difference between the two results is the location in which those results are stored, either the Server's postgres db or a local file path. So I think your best option is to use
LocalResult
as it would be available to both flows because it caches to a local directory, or potentially a custom class though I'm not sure what that would look like. Maybe you could parameterize/setup some conditional logic for your Result type to define at run time, depending on whether you are doing a local run or a registered run. 🤔
j
Hi @Kyle Moon-Wright, thanks for your reply. We already tried using
LocalResult
on prefect server once. It worked, but since you're referring to this case, I'd mention one minor inconvenience. By default, it stores results to
~/.prefect/results
. When you do
workflow.register(...)
on your local PC,
~
gets resolved to your local home directory before registering the workflow. So the server expects path to be exactly the same as on the local PC, instead of resolving
~
on the server and using servers's home dir. But I know I can bypass this by setting some concrete path and make it the same as on the server. I'd otherwise be interested in how other people do some unit testing of workflows and if others also have the need for local/registered run differentiation. One reason why we need it is also debugging. Are there some better options for such cases?
k
This is great feedback, let me see if I can find some examples or documentation on this.
Though I wasn't able to find common practices detailing how to test between local/registered runs, we do have information on unit testing in the documentation here and here. Would love to hear how others may be testing between local/registered runs and best practices they've discovered!
j
Thank you for the advice. We will probably resort to
LocalResult
for now.