Hi! I have an ETL flow that utilizes mapping to process each file in a separate branch. The transformation is done in multiple steps and in one of the steps I need to be able to access the input arguments of an upstream task. Example: extract_file -> apply_transformation_1 -> apply_transformation_2 -> apply_transformation_3 (here I need to access input arguments of apply_transformation_1) -> load. Does Prefect offer some built-in functionality for implementing this kind of a "task context" or do I need to create a custom class that contains task arguments and its results and is passed between tasks? Thanks in advance!

Hi @Simo Tumelius the sort of task context you are describing is something that only exists in the context of a single task’s run. (i.e. a task’s context is not passed to any other tasks) I think your best bet to accomplish what you are after is to take whichever inputs that are passed into

apply_transform_1

and pass them through to

apply_transform_3

. Or in a different sense take the results of

extract_file

(which are the inputs to transform_1) and pass them directly to the transform_3 in the downstream

Got it, thanks @josh!