Marwan Sarieddine
12/28/2020, 2:31 PMprefect.context
in the cluster_class
callable and the modify the context when running the flow (not ideal but I can't think of another approach)
EDIT - after some thought - modifying the context dynamically in the flow run to modify the adapt kwargs perhaps isn't as bad as I first thought (will have to wrap the flow with an outer flow basically)...
It would be nice to get a confirmation if this is the best practice right now...Zanie
Marwan Sarieddine
12/28/2020, 3:37 PMZanie
Jim Crist-Harif
12/28/2020, 8:11 PMcluster_class
is the current recommended solution. This should give you access to everything in prefect.context
to create your cluster. What info do you need to dynamically make decisions here?Marwan Sarieddine
12/29/2020, 12:21 AMZanie
prefect.context.parameters
Marwan Sarieddine
12/29/2020, 12:23 AM