Josh Greenhalgh
03/17/2021, 9:18 PMValueError: Tasks with variable positional arguments (*args) are not supported, because all Prefect arguments are stored as keywords. As a workaround, consider modifying the run() method to accept **kwargs and feeding the values to *args.
Has anyone had any luck using their own? The one I want to use is this;
def jitter(func: Callable) -> Callable:
"""
For use with short running mapped tasks
to avoid overwhelming the api server
calls function sleeps for random period
"""
def wrapped(*args, **kwargs):
res = func(*args, **kwargs)
sleep(random.gauss(1.2, 0.2))
return res
return wrapped
Basically just waits for a while and does so randomly so lots of short running mapped tasks don't all hit my api server (on k8s) - I much prefer this than having sleeps in my actual task codeJosh Greenhalgh
03/17/2021, 9:19 PMJim Crist-Harif
03/17/2021, 9:20 PMdef wrapped
with functools.wraps
import functools
def jitter(func: Callable) -> Callable:
"""
For use with short running mapped tasks
to avoid overwhelming the api server
calls function sleeps for random period
"""
@functools.wraps(func)
def wrapped(*args, **kwargs):
res = func(*args, **kwargs)
sleep(random.gauss(1.2, 0.2))
return res
return wrapped
Josh Greenhalgh
03/17/2021, 9:22 PMJosh Greenhalgh
03/17/2021, 9:22 PMJim Crist-Harif
03/17/2021, 9:23 PMJosh Greenhalgh
03/17/2021, 9:23 PM