Dumb question. I am just getting started and was w...
# prefect-getting-started
r
Dumb question. I am just getting started and was wondering how I could see the inputs and outputs of both flows and task runs. (please let me know if this is the wrong channel). Here is the file being run:
Copy code
from prefect import flow, task
from prefect.filesystems import LocalFileSystem

@flow(persist_result=True, result_storage=LocalFileSystem(basepath=".prefect"), result_serializer="json")
def do_stuff():
    do_a(1, 2, 3)
    do_b('b')
    return "hello there"

@task(persist_result=True)
def do_a(a, b, c):
    sum = 0
    for i in range(1000000):
        sum += i
    
    return sum

@task(persist_result=True)
def do_b(b):
    print("running b")
    return "hello world"

if __name__ == "__main__":
    do_stuff()
The UI shows the location where the result is persisted, but I was wondering if I could somehow see the results in the UI.
The inputs are also empty, aside from the name of the variables passed in.
j
This is mostly a feature - prefect doesn't upload things to the cloud interface until specifically told to (your inputs could have confidential data). Bare persisted results are not fun to look at - they are base64 encoded if you look at that file so you'll need to write code to even make sense of them.
r
That's what I figured based on the architecture (for not showing the results). Is there anyway to turn it on? I am planning to use everything locally on a system more for tracking pipeline runs.
j
I wouldn't use prefect's persisted results for your own purposes - this feature is for prefect to implement caching to avoid retrying tasks unnecessarily. IMHO, you should save your own data in places that you control using a method appropriate for your problem space.
r
Sounds good. Thank you! I wanted to use Prefect personally just to learn it, but it doesn't seem to align with the current use case at my work. Have a great evening!
j
You can put stuff in the UI by creating "artifacts" if that helps.
r
I saw that, but it felt like overkill...but that's not a bad idea now that I think about it some more. šŸ¤”
a
@Richard Chang out of curiosity what is the current use case at work?
n
hey @Richard Chang! (not to distract from adam's question šŸ™‚ also curious to hear about your use case) if you were thinking about exploring Artifacts, here's a (silly but perhaps relevant) example of how you can use them to selectively render information in the UI. Generally we store params sent to entrypoints of deployments, but not nested runs' parameter values
a
Thought I will write in here cause that's exactly my case right now - so if I want to pass e.g. DB records somewhere outside Prefect - best way would be to use artifact and API call?
n
> so if I want to pass e.g. DB records somewhere outside Prefect hmm artifacts are broadly "for" rendering some visual of your runtime data that you can see in the prefect UI im not sure what you mean by db records or "outside Prefect" exactly, but I'm not sure if that sounds like artifacts to me - for example, lots of our users often want to just write/read values to s3 directly themselves (with either boto3 or an S3Bucket block), which makes a lot of sense in the course of normal DE workflows let me know if Im misunderstanding something about your question!
a
I see, basically I have external app that triggers the flow through an API, that flow gets some records from DB. I was just thinking what would be the best way to get that fetched records back to the app that triggered the flow. Couldn't find any API endpoint I could use for that
r
In my case, we are doing an AI-related workflow that requires multiple passes of traditional ML and LLMs, but we keep having bugs and unpredictable results. It's hard to locate where it's happening, so I was thinking about using Prefect to capture the inputs, outputs, and run time of each function. We have a somewhat monstrous function that is hard to debug, and things are supposed to get more complicated from there. I love the
@task
and
@flow
decorators though and how unintrusive it is. Most of the people are my team are not software engineers, so it is nice to have minimal lift.
@Nate, thanks for taking the time to even write out a code sample. It's very instructive as someone who is completely new.
catjam 1
n
We have a somewhat monstrous function that is hard to debug, and things are supposed to get more complicated from there.
@Richard Chang I'm glad you found prefect, as this sounds like a good fit! lots of folks will come with a big old python script, that they incrementally add in those task / flow decorators into as they need retries / caching etc theres a couple things I think you might be interested in based on this:
requires multiple passes of traditional ML and LLMs ... bugs and unpredictable results
• the transactions semantics outlined in this post, as they allow you to define a series of outcomes you need, that you only want to commit to if they all work, and you can rollback if some dont (ie LLM presented with query -> does a bunch of work that might fail making side affects-> produces result) • https://controlflow.ai/welcome and https://github.com/PrefectHQ/marvin (for agentic workflows / structured outputs respectively)