is the return from another task. When I execute this I'm getting
Copy code
TypeError: mount_glue_catalog() got an unexpected keyword argument 'upstream_tasks'
When I call this from a flow context, the invocation of
mount_glue_catalog
doesn't have any issues. Does the interface to running this task somehow change based on my calling it with run() vs from within a flow context?
k
Kevin Kho
09/28/2021, 5:59 PM
So I don’t know for sure if you can use
Task
kwargs when calling run like this, but in general I don’t think you can because using a task like this is just the underlying Python function. With
upstream_tasks
specifically, there is a disconnect where
upstream_tasks
is a build-time concept as the DAG is created and using a task in another task is a run-time thing and you can’t create the dependency during runtime. The
upstream_tasks
should go in the
Flow
block
c
Constantino Schillebeeckx
09/28/2021, 6:24 PM
cool got it - as always, thanks for the speedy response
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.