hey everyone, is there a way to use variables that are global to every task like an increment counter ?

Good question! you should consider task runs as stateless. In fact, if you leverage DaskExecutor, then each of your task runs can run on a different machine, that's why sharing state between tasks is hard and impractical you can share some global static variable or pass some value to downstream tasks as data dependency e.g. from a parameter value, but the increment counter-example won't work reliably across executors and various environments

ok thanks, i'll do a sort of map reduce then