Ben Muller

    Ben Muller

    1 year ago
    Hey peeps, What am I doing wrong here. I am trying to provide multiple clocks into my flow schedule ( trying with one to start off with ) but running into an error on the param. I have followed other examples, but it just isnt working here
    schedule = Schedule(
        clocks=[
            CronClock(
                cron="*/7 * * * *",
                parameter_defaults=dict(account_identifier="inrun"),
                start_date=datetime.now(tz=timezone("Australia/Brisbane")),
            )
    )
    
    with Flow(
        name="betfair_flow",
        storage=Storage().in_a_s3_bucket(),
        run_config=RunConfig().fargate_on_ecs(cpu=512, memory=2048),
        schedule=schedule,
        executor=LocalDaskExecutor(scheduler="threads"),
    ) as flow:
        betfair_account_identifier = Parameter("account_identifier")
    Error in 🧵
    raise ValueError(
      ValueError: Flow.run did not receive the following required parameters: account_identifier
    FYI setting it as
    parameter_defaults={"account_identifier": "inrun"},
    also does not work
    Oh, and I'm trying to run this flow from the cli.
    Jacob Blanco

    Jacob Blanco

    1 year ago
    Can you show how you’re actually running the flow? or are you waiting for the schedule to run??
    Ben Muller

    Ben Muller

    1 year ago
    prefect run flow -p path/to/flow
    And yes, I'm waiting for the schedule
    Kevin Kho

    Kevin Kho

    1 year ago
    Parameters are not registered unless they are used in the Flow. Only tasks that are used in the flow are registered and doing
    betfair_account_identifier = Parameter("account_identifier")
    only calls the
    init
    but not the
    run
    . You can either use this somewhere downstream or you might be able to do
    Parameter("account_identifier")()
    Ben Muller

    Ben Muller

    1 year ago
    I'm a bit confused @Kevin Kho. I actually haven't registered this flow yet. I'm just trying to run it locally to make sure it works.
    And the param is used in the flow, I just removed the logic to show a shorter post.
    Kevin Kho

    Kevin Kho

    1 year ago
    Do you use
    betfair_account_identifier
    somewhere in the flow?
    Ben Muller

    Ben Muller

    1 year ago
    Yes
    It is vital, hence why I'm using it as a parameter
    Kevin Kho

    Kevin Kho

    1 year ago
    How do you call
    flow.run()
    ? Do you pass it in or you’re relying on the default?
    Ben Muller

    Ben Muller

    1 year ago
    I'm running it from the cli
    Kevin Kho

    Kevin Kho

    1 year ago
    Do you pass
    --param key=value
    ?
    Ben Muller

    Ben Muller

    1 year ago
    No, I assumed that it picks up the schedule, like it usually does with the cli?
    If I run a flow with the cli it says waiting for scheduled run, why would it not pick up the param then too?
    Kevin Kho

    Kevin Kho

    1 year ago
    Ahh I think this is a bug where default parameters don’t satisfy required parameters. Not 100% sure yet. I think in the mean time you can make that Parameter not required
    Ben Muller

    Ben Muller

    1 year ago
    I think when I made it not required it didn't work either. It's the middle of the night here. Will test in the morning. As far as I can see it wouldn't pick up the default params dict when running from the cli
    Kevin Kho

    Kevin Kho

    1 year ago
    ok will test.
    requiired=False
    works for me but I will explore teh issue more
    Ben Muller

    Ben Muller

    1 year ago
    Yeah, it ran but the param value is None. Not what you expect in the clock, right?
    Kevin Kho

    Kevin Kho

    1 year ago
    I got the default value actually O.o
    from prefect import task, Flow, Parameter
    from prefect.schedules import Schedule 
    from prefect.schedules.clocks import CronClock
    from datetime import datetime
    import prefect
    
    schedule = Schedule(
        clocks=[
            CronClock(
                cron="* * * * *",
                parameter_defaults=dict(account_identifier="inrun"),
                start_date=datetime.now(),
            )]
    )
    
    @task
    def abc(x):
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>(x)
        return x
    
    with Flow(
        name="betfair_flow",
        schedule=schedule,
    ) as flow:
        betfair_account_identifier = Parameter("account_identifier", required=False)
        abc(betfair_account_identifier)
    
    flow.run()
    Ben Muller

    Ben Muller

    1 year ago
    Hmm, but running from the cli is how I'm doing it. Can you please confirm you get that, too?
    Kevin Kho

    Kevin Kho

    1 year ago
    Ok I am the same as you now. My gosh I don’t know if this is two separate bugs
    Ben Muller

    Ben Muller

    1 year ago
    Just an idea, is it with the newer version of prefect, I've seen floating around we might have to use that flag
    --with-schedule
    or something like that
    that didnt work
    Kevin Kho

    Kevin Kho

    1 year ago
    You still up? lol I was gonna reply later but yeah I think there is legit at least one bug here. Will write up the issue later
    Ben Muller

    Ben Muller

    1 year ago
    I just woke up, it is 8am here lol
    you always get my timezone messed up 😆
    Kevin Kho

    Kevin Kho

    1 year ago
    I feel like you just said it was the middle of the night. I guess you barely sleep 😆
    Ben Muller

    Ben Muller

    1 year ago
    It was before lol. You can check your slack messages when you have an 8 month old son who never sleeps! 😂
    Hey @Kevin Kho, did we end up deciding this is a bug?
    I have had to run with cli command providing params. I guess that's kind of expected, but maybe it should be documented
    Kevin Kho

    Kevin Kho

    1 year ago
    What timing. Literally writing the ticket right now
    Ben Muller

    Ben Muller

    1 year ago
    😂
    Do you ever take a break?
    You're a 🤖
    And thanks for the help on it!
    Kevin Kho

    Kevin Kho

    1 year ago
    I did…that’s why it’s delayed lol. But I work on other open source stuff on the weekend haha
    Ben Muller

    Ben Muller

    1 year ago
    Nice, thanks so much.
    Kevin Kho

    Kevin Kho

    1 year ago
    There was a response here btw
    Ben Muller

    Ben Muller

    1 year ago
    I agree with the comment. Good work @Kevin Kho