https://prefect.io
Join Slack
Hi, I'd like to understand the logic of data logging a bit more. Below is the flow I'm running, and ...
# ask-community
a
Hi, I'd like to understand the logic of data logging a bit more. Below is the flow I'm running, and I thought that the second task would log the input value (as it's json-serialisable) as 
task_inputs
field of 
task_run
table of the DB. But instead I find only 
{"name": [{"input_type": "task_run", "id": "c8ab67da-0eda-4a0d-9fa8-fb89232adbfe"}]}
in that field. Are task inputs logged somewhere else? Using prefect v2.19.9.
Here's the flow I run:
Copy code
from prefect import flow, task


@task
def get_string() -> str:
    return "World"


@task
def say_hello(name: str) -> None:
    print(f"Hello, {name}!")


@flow(log_prints=True)
def min_flow() -> None:
    say_hello(get_string())


if __name__ == "__main__":
    min_flow()
n
hi @Anton Buyskikh - we *almost never store task run parameters in the API (in contrast to flow run parameters), only references to upstream inputs that come from other task runs • almost because we do store 
bool
and 
int
values since they can't really be sensitive by themselves
this is for a couple reasons: • security (lots of PII is loaded into flows and passed between tasks, for example) • lots of times, task run parameters are not json serializable
if you want to log task run parameters, you can, but you'd be responsible for surfacing or storing those values somehow
a
Hi @Nate thank you for the quick answer and revealing the reason! That's all good, I shall log arguments via other methods then, that's not a problem.
👍 1
7 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by