https://prefect.io logo
Title
s

Stefan

09/12/2022, 11:34 AM
Looking for solutions on how to "call a task from within a task" • My main task, "Get data from sql" will get the data. • Within the above function, I run another function to generate the SQL-statement with the specifics sent into the parent function (such as a date from a list of datetimes). • Since I cannot decorate the sql-generating function with a @Task - how can I run it and still see it in Prefect? I see from the docs that my options are either 1) use sql.fn() or 2) not decorate it with a @Task - neither of which will not generate a task run. Here is a mockup: @task
def sql_builder(date):
statement = statement generator with date and tables
return statement
@task
def get_data(date):
sql = sql_builder(date)
get_data(sql, con)
1
t

Tony Piazza

09/12/2022, 12:54 PM
@Stefan why not make the get_data function a flow instead? your flow can call it directly as a subflow.
s

Stefan

09/12/2022, 12:57 PM
Thank for your reply. My first thought was that would seem pretty overkill to generate that amount of Flow-runs in addition to the main one, especially if I'm doing a year's worth of data (one subflow per day) (imagine the UI)
t

Tony Piazza

09/12/2022, 1:19 PM
curious why do you need to "see it in Prefect" ?
s

Stefan

09/12/2022, 1:29 PM
To take advantage of what Prefect has to offer for tasks? See output, use retries, map-function, tags.
t

Tony Piazza

09/12/2022, 1:30 PM
then it has to be either a task or flow. will be interested to see how you decide to implement it.
b

Bianca Hoch

09/12/2022, 10:11 PM
Hi Stefan, we wouldn't recommend using
.fn()
due to the fact that you lose all orchestration functionality that prefect tasks have. You can, however, have a flow call the task, and a flow call a flow.
s

Stefan

09/14/2022, 10:11 AM
Thanks @Bianca Hoch!