Anish Chhaparwal

10/24/2020, 11:34 PM
I have a python program with large number on input parameters. say around 35+. We are currently using argparser to set default values, load some of them from config files and pass them all args to functions. I'd like to convert the program into a flow. Is there a way to use the existing args in the flow. I have illustrated a sample of what id like to achieve but this fails when i try to run this from UI (refer attach image too).
import argparse
from prefect import Flow, Parameter, task
import prefect
from prefect.environments import LocalEnvironment
def testing(args):
logger = prefect.context.get("logger")
<|>(f"agrs value for name is : {}")
<|>(f"agrs value for worker is : {args.workers}")
with Flow("args_trial") as flow:
opts = Parameter("opts")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="test pywinrm")
parser.add_argument("--workers", type=int, default=10)
parser.add_argument("--sleep_time", type=int, default=2)
parser.add_argument("--name", type=str, default="args_tester")
opts = parser.parse_args()
flow.environment = LocalEnvironment(labels=["qure9", "pipelines"])
I can seem to figure out how to pass the default args while registering. Also, if i have to convert each of them into prefect parameters, i'd like to pass all parameters to a function using something like def func_1(args) instead of specifying 15-20 to each function. is that possible?

Chris White

10/25/2020, 9:07 PM
Hi Anish - can you please edit your main comment and post the code block within a thread in the future? That way the main channel stays tidy. Otherwise, I’m not sure what you’re asking for here - you can provide parameter defaults to the
constructor like
opts = Pararameter("opts", default=some_default_value)
and for flows that require large amounts of parameters, you might consider combining them all into a single dictionary

Anish Chhaparwal

10/26/2020, 7:55 PM
@Chris White will keep that in mind the next time. Didnt realize you can pass dictionaries in parameter. Thanks.
👍 1
The only drawback being i wont be able to give any parameter dtypes or ensure some of them are mandatory.