Hi. I'm new to Prefect and having some trouble understanding the concept of tasks and task runs. I'm trying to return data from a task, but attempting to print the results of the task only prints out
<Task: fetch_data>
. This is what I'm trying to do:
@task
def fetch_data():
return {"data": "random data"}
with Flow('Get-Data') as flow:
flow.run_config = LocalRun()
data = fetch_data()
<http://logger.info|logger.info>(data)
flow.run()
k
Kevin Kho
04/13/2022, 2:29 AM
Hey @Carlos Cueto,
This is because
<http://logger.info|logger.info>
is logging during the flow build time, not during runtime. Also, your run_config should be outisde the flow
Copy code
with Flow('Get-Data') as flow:
data = fetch_data()
<http://logger.info|logger.info>(data)
flow.run_config = LocalRun()
So the
with Flow()
builds the DAG and the logger is running during buildtime
c
Carlos Cueto
04/13/2022, 2:53 AM
Thank you Kevin. I'm still a little confused. I think I understand your point. Can you advise how to print the results of
fetch_data
during flow runtime and not build time? Would I be forced to call the
<http://logger.info|logger.info>()
inside a
@task
decorated function?
In addition, do you mind explaining the difference between run_config inside the flow vs outside the flow context?
k
Kevin Kho
04/13/2022, 3:49 AM
Yes exactly use the logger inside the task
Kevin Kho
04/13/2022, 3:51 AM
The
Copy code
with Flow('Get-Data') as flow:
...
build the flow and then the
Copy code
flow.run_config = ...
sets a property.
It may work, but there are some edge cases where this will cause some weirdness. The Flow block is really intended to build the DAG
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.