Is there a way to "package" up a set of tasks into a super task that can be re-used easily? For example, we have a few tasks that usually get used in sequence: validate some pandas dataframe, grab some secrets from AWS, use those secrets/credentials to insert the dataframe into a database

Could you use a subflow?

ya, i guess i could. but it seems like more overhead

Yeah, I'd prefer that too. Maybe there will be additional UI features related to subflows in the future, that'd probably make them a good fit for this kind of use case. I can't think of any other ways of solving your problem, since you can't call tasks from tasks.

Subflows are probably the literal solution to your question, as they create a single “virtual task” in the parent flow for all their component tasks but still let you “drill down” to the component tasks if you want. Avoiding subflows, two thoughts occur to me: 1. you could distribute your tasks as part of a larger function (just a regular function, not a flow) that you call inside your flows. This would then pass the inputs to your tasks in sequence. This aligns with your goal of seeing all of your tasks in the UI, but only calling one function. 2. if your goal is to package them into a single task (still without a subflow), you should be able to recover their underlying functions as

your_task.fn(…)

. This aligns with your goal of packaging them as a super task (which could have its own retry settings, for example), but would not let you see them individually in the UI.

Interesting @Jeremiah . What would 1 look like

Something like this --

ok cool. can i do that in v1? I was under the impression that everything in the flow must be a task

Yes, I expect this would work in V1 — every computation in the flow must be a task, but here we’re using

super_task

purely as a convenience function; it’s equivalent to calling

task_1

and

task_2

yourself