hey everyone, is there a way to use variables that are global to every task like an increment counter ?
in prefect 1
06/09/2022, 1:25 PM
Good question! you should consider task runs as stateless. In fact, if you leverage DaskExecutor, then each of your task runs can run on a different machine, that's why sharing state between tasks is hard and impractical
you can share some global static variable or pass some value to downstream tasks as data dependency e.g. from a parameter value, but the increment counter-example won't work reliably across executors and various environments
main point: you shouldn't rely on any shared state between task runs and leverage data dependencies to pass data between tasks