@Marvin In my orchestrated pipeline I want to retrieve a value from a database in a task to be reused in following tasks of that flow. Is there a prefect-way to redefine that variable in each flow run, store that value as a variable and pass it onto tasks of this flow? So I guess this should not be confused with environment variables.

Certainly, in Prefect, you would achieve this by using the return values of tasks to pass data between tasks within a flow. Here is a simplified example of how you can retrieve a value from a database in one task and use it in other tasks:

python
from prefect import flow, task

@task
def get_database_value():
    value = ...  # retrieve from database
    return value

@task
def use_value(value):
    # Perform some operation with the retrieved value
    result = value + 10  # example operation
    return result

@flow
def my_flow():
    db_value = get_database_value()  # Retrieve the value within the flow
    processed_value = use_value(db_value)  # Pass the retrieved value to another task
    return processed_value

if __name__ == "__main__":
    my_flow()

Here,

get_database_value

is a task that retrieves a value from a database, and

use_value

is a task that uses that value. The return value from the first task is passed to the second task within the flow function

my_flow

. This is how you pass data between tasks in a flow with Prefect. If you need the value to vary for each flow run and be dynamic, ensure that your database query or retrieval logic within the

get_database_value

task accounts for this. For more information on passing data between tasks and creating Prefect flows, you can refer to this example from the Prefect GitHub repository: Prefect Tasks Example.