ale
11/05/2020, 8:58 AM@task
support both name
and task_run_name
at the same time?
It seems that name
takes precedence over task_run_name
, am I right?ale
11/05/2020, 9:06 AMtask_run_name
https://docs.prefect.io/core/idioms/task-run-names.html#naming-task-runs-based-on-inputsemre
11/05/2020, 9:38 AMname
param is a prefect core feature, and this name is displayed on things like the prefect generated logs for your flow run, and flow.visualize()
. It should help you understand a flows structure and what each task does in a high level visualization.
The task_run_name
is I believe a prefect cloud or prefect server feature, and is used to tag a specific task run with a human readable name. If you do not set a task_run_name
for a task, you should see that in different runs of the same flow, the same task gets a random task run name each time. Two word stuff like, lunatic-landlord
or emotional-alligator
.
I haven’t ever set a task_run_name
explicitly, but I believe setting it would use the same value for that tasks task run name between each flow run.emre
11/05/2020, 9:41 AMtask_run_name
is redundant when running without a backend, because when running without a backend, your flow runs and task runs aren’t persisted anywhere. You don’t need a human readable name for a task run if it isn’t going to be stored and later on read by a human.ale
11/05/2020, 10:05 AMale
11/05/2020, 11:35 AMtask_run_name
as described in the above doc.
But when running flows from Prefect UI I don’t see the values from computed with task_run_name
.
I see the task method name instead.
My task is defined as:
@task(task_run_name=lambda **kwargs: f"Create table {kwargs['table_name']} in DWH")
def create_table(engine, table_name, drop_if_exists):
db_utils.create_table(
engine=engine,
table_name=table_name,
drop_if_exists=drop_if_exists
)
Any suggestions?Julian
11/05/2020, 1:32 PMJulian
11/05/2020, 1:33 PMlogger = prefect.utilities.logging.get_logger()
task_run_id = prefect.context.task_run_id
updated = Client().set_task_run_name(task_run_id, 'test name')
<http://logger.info|logger.info>("Updated task_run_name? %s", updated)
and it tells me that the task_run_name was updatedale
11/05/2020, 1:38 PMnicholas
task_run_name
is a concept that only exists in Cloud/Server since it requires some sort of persistence.
There are two objects being discussed here: task and task run; the first is the blueprint you give to prefect in the form of an extended or decorated task class, and the second is the actual instantiation (the run) of that task. When you provide a task with a name
, you're naming its blueprint, but when you give it a task_run_name
, you're templating its instantiation. The UI handles this in a few ways, both for backwards compatibility (task run names were removed as a column in the db early in 2020 but were reintroduced ~last month) and for readability/context: anything describing a task
or flow
will show the blueprint task name. Anything describing a run (task_run
or flow_run
) will show the templated task_run_name
if it exists or will show a combination of the task
(blueprint) name and its parent flow_run
name. Since older versions of Cloud/Server didn't have task run names in the db, older versions of the UI won't show task run names.ale
11/05/2020, 4:05 PMCreate table <table_name> in DWH
.
But what I actually see is create_table
.
The flow runs in Prefect Server.
What am I missing?nicholas
ale
11/05/2020, 4:11 PMnicholas
import prefect
from prefect import Flow, task, Parameter
@task(task_run_name=lambda **kwargs: f"Create table {kwargs['table_name']} in DWH")
def create_table(table_name):
# create some table
return
with Flow("Task Run Names") as flow:
table_name = Parameter("table_name", default="user")
create_table(table_name=table_name)
nicholas
ale
11/05/2020, 4:19 PMale
11/05/2020, 4:19 PMnicholas
ale
11/05/2020, 4:21 PMnicholas
ale
11/05/2020, 4:35 PMnicholas
ale
11/05/2020, 4:35 PMale
11/05/2020, 5:01 PMnicholas