< Marvin> Is it possible to dynamically name a task I have a Prefect Community #ask-marvin

<@ULVA73B9P> Is it possible to dynamically name a ...

Colton

05/21/2024, 9:24 PM

@Marvin Is it possible to dynamically name a task? I have a defined task/function that gets executed multiple times in a flow. I Want the task in the UI to have a descriptor that includes information on the input arguments, is this possible?

Marvin

05/21/2024, 9:25 PM

Yes, it is possible to dynamically name a task in Prefect. You can achieve this by defining a function that returns a string to be used as the task run name and then using the

task_run_name

parameter when defining your task. Here are examples to guide you: 1. Basic Dynamic Name:

Copy code

python
   import datetime
   from prefect import flow, task

   def generate_task_name():
       date = datetime.datetime.now(datetime.timezone.utc)
       return f"{date:%A}-is-a-lovely-day"

   @task(name="My Example Task", description="An example task for a tutorial.", task_run_name=generate_task_name)
   def my_task(name):
       pass

   @flow
   def my_flow():
       # Creates a run with a name like "Thursday-is-a-lovely-day"
       my_task(name="marvin")

   my_flow()

2. Dynamic Name with Task Information:

Copy code

python
   from prefect import flow, task
   from prefect.runtime import flow_run, task_run

   def generate_task_name():
       flow_name = flow_run.flow_name
       task_name = task_run.task_name
       parameters = task_run.parameters
       name = parameters["name"]
       limit = parameters["limit"]
       return f"{flow_name}-{task_name}-with-{name}-and-{limit}"

   @task(name="my-example-task", description="An example task for a tutorial.", task_run_name=generate_task_name)
   def my_task(name: str, limit: int = 100):
       pass

   @flow
   def my_flow(name: str):
       # Creates a run with a name like "my-flow-my-example-task-with-marvin-and-100"
       my_task(name=name)

   my_flow(name="marvin")

These examples show how you can dynamically assign names to task runs based on the current date or the input arguments of the task. You can customize the function to include other relevant details as needed. For further reference, you can check the Prefect tasks documentation.

🙌 1

Colton

05/23/2024, 6:53 PM

@Marvin is it possible to make a generate_task_name() more universal, something like this: @task(name='refreshSpiceLayer', task_run_name= f"Refresh QS dataset: {getArgValueFromTask('dataset_name')}") where def getArgValueFromTask(arg_name): parameters = task_run.parameters arg_value = parameters[arg_name] # parameter names must match args name in function return arg_value

Marvin

05/23/2024, 6:53 PM

Yes, you can create a more universal

generate_task_name

function and use it in your Prefect tasks. Here’s how you might achieve that: 1. Define a Universal
generate_task_name
Function:

Copy code

python
    from prefect import flow, task
    from prefect.runtime import task_run

    def get_arg_value_from_task(arg_name):
        parameters = task_run.parameters
        return parameters[arg_name]

    def generate_task_name():
        dataset_name = get_arg_value_from_task("dataset_name")
        return f"Refresh QS dataset: {dataset_name}"

2. Use it in Your Task Definition:

Copy code

python
    @task(name='refreshSpiceLayer', task_run_name=generate_task_name)
    def refresh_spice_layer(dataset_name):
        pass

3. Create and Run Your Flow:

Copy code

python
    @flow
    def my_flow(dataset_name: str):
        refresh_spice_layer(dataset_name=dataset_name)

    my_flow(dataset_name="example_dataset")

This setup should dynamically generate a task name based on the argument values. Note that the

task_run.parameters

object provides access to the parameters passed to the current task run. Here is a summary of the necessary documentation references: - Prefect Tasks - Prefect Runtime These resources provide further details on defining tasks, accessing runtime variables, and dynamic task naming.

90 Views

Open in Slack

Previous Next