Richard Chang
08/27/2024, 11:43 PMfrom prefect import flow, task
from prefect.filesystems import LocalFileSystem
@flow(persist_result=True, result_storage=LocalFileSystem(basepath=".prefect"), result_serializer="json")
def do_stuff():
do_a(1, 2, 3)
do_b('b')
return "hello there"
@task(persist_result=True)
def do_a(a, b, c):
sum = 0
for i in range(1000000):
sum += i
return sum
@task(persist_result=True)
def do_b(b):
print("running b")
return "hello world"
if __name__ == "__main__":
do_stuff()
The UI shows the location where the result is persisted, but I was wondering if I could somehow see the results in the UI.Richard Chang
08/27/2024, 11:49 PMJanet Carson
08/28/2024, 12:03 AMRichard Chang
08/28/2024, 12:05 AMJanet Carson
08/28/2024, 12:08 AMRichard Chang
08/28/2024, 12:09 AMJanet Carson
08/28/2024, 12:11 AMRichard Chang
08/28/2024, 12:12 AMAlexander Azzam
08/28/2024, 6:13 AMNate
08/28/2024, 11:30 AMAdalbert
08/28/2024, 12:12 PMNate
08/28/2024, 2:46 PMAdalbert
08/28/2024, 3:58 PMRichard Chang
08/28/2024, 4:01 PM@task
and @flow
decorators though and how unintrusive it is. Most of the people are my team are not software engineers, so it is nice to have minimal lift.Richard Chang
08/28/2024, 4:03 PMNate
08/28/2024, 4:35 PMWe have a somewhat monstrous function that is hard to debug, and things are supposed to get more complicated from there.@Richard Chang I'm glad you found prefect, as this sounds like a good fit! lots of folks will come with a big old python script, that they incrementally add in those task / flow decorators into as they need retries / caching etc theres a couple things I think you might be interested in based on this:
requires multiple passes of traditional ML and LLMs ... bugs and unpredictable results⢠the transactions semantics outlined in this post, as they allow you to define a series of outcomes you need, that you only want to commit to if they all work, and you can rollback if some dont (ie LLM presented with query -> does a bunch of work that might fail making side affects-> produces result) ⢠https://controlflow.ai/welcome and https://github.com/PrefectHQ/marvin (for agentic workflows / structured outputs respectively)