Thread
#prefect-community
    c

    Charles Liu

    1 year ago
    Hi all! I'm looking to leverage Prefect Parameters in order to sometimes manually enter run parameters if a pipeline needs to be re-run. I'm looking to clarify what exactly the difference between Context and Parameters is and how to use them correctly as I'm a little confused by the documentation.
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Charles Liu, I think if you are passing in data, it should be in the form of Parameters. The context is more meant to hold these things that set at runtime (flow name, today, task id). The parameters are under the context. Context is more of configurations around the flow. You could use it though for stuff like environments. Like this from the docs.
    environment = "prod"
    user = "${environments.${environment}.user}"
    
    [environments]
    
        [environments.dev]
            user = "test"
    
        [environments.prod]
            user = "admin"
    If you have this in the config.toml, you can then check the value inside your Flow. So think of these as settings you want on the agent also, whereas parameters are really about the Flow itself agnostic to where it is running. There is some overlap in what can be achieved with parameters an the context though.
    c

    Charles Liu

    1 year ago
    Okay that does clarify things! Definitely not trying to use Context in this instance then.
    What I'm actually trying to accomplish is override an integer. I know there's the --parameter-string for CLI, but I also noticed there's a Parameters window in Settings on the flow page on the front end. Is there something I need to do to activate that window? Currently it says my flow has no default Parameters but also does not give me an empty json window like in the docs.
    Kevin Kho

    Kevin Kho

    1 year ago
    Yeah I think you need to register your flow with Parameters
    x = Parameter("x", default=1)
    Then you should see that window populated. And then you can specify the default there and override this. But if you just want one run with a different Parameter value, you should click the
    Run
    instead of
    Quick Run
    and it will let you set the Parameters there for that individual run.
    c

    Charles Liu

    1 year ago
    okay so is this the right way to use it for example:
    important_number = Parameter("important_number", default=3)
    
    @task()
    def this_task():
       task_params:{
           "important_number": 3
            }
    
    with Flow()...etc.
    Kevin Kho

    Kevin Kho

    1 year ago
    @task
    def abc(x):
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>(x)
        return x
    
    with Flow("context-test") as flow:
        x = Parameter("x", default=3)
        a = abc(x)
    c

    Charles Liu

    1 year ago
    ah okay it has to go in the flow block gotcha
    Kevin Kho

    Kevin Kho

    1 year ago
    The task will just treat it as a regular input. The Parameter is a special task.
    c

    Charles Liu

    1 year ago
    ahhhh okay that makes it make a little more sense when you put it that way
    Say I want to use timedelta along with Parameters, is there a way to access the Parameters object in the flow block?
    Kevin Kho

    Kevin Kho

    1 year ago
    What do you mean by access? Do you mean something else other than passing it to another task?
    c

    Charles Liu

    1 year ago
    Hmm you bring up a good point I could just operate on the value within a task and pass out the result. Basically the integer I want to edit helps create another value; and that value I want to set as the default of another Parameter so that it defaults to an updated value every time.
    Although having that logic mushed into another task or having its own task seems unwieldily for such a small step? Unless that's best practice to taskify every little thing.
    Kevin Kho

    Kevin Kho

    1 year ago
    Oh you probably need a task for that. Yes most things in the Flow block need to be a task but you can make a lambda to make it faster.
    task(lambda x: x+1)(value)
    c

    Charles Liu

    1 year ago
    !!! will give that a shot
    Kevin Kho

    Kevin Kho

    1 year ago
    I realized I don’t think you can do that where you do:
    with Flow("context-test") as flow:
        x = Parameter("x", default=1)
        y = Parameter("y", task(lambda x: x + 1))
        a = abc(y)
    The default needs to be static because it’s registered and saved during build time.
    y
    here shouldn’t be a Parameter. You can just do the
    y = task(lambda x: x +1)(x)
    c

    Charles Liu

    1 year ago
    Understood, will let you know if it works out
    Okay so followup question: how do you unregister a parameter?
    I removed one from my flow block and it's still there on the front end
    Kevin Kho

    Kevin Kho

    1 year ago
    Even when you re-register the flow? Did you try checking the latest version?
    c

    Charles Liu

    1 year ago
    I believe the version number is incrementing but the removed Parameter() is still on the front end Run page.
    The way I'm re-registering my flow is running the .py file
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey I’ll test this now
    You mean here right?
    It is working for me (I re-registered with less parameters)…could you try it on a separate flow and see if it still lingers in the UI?
    c

    Charles Liu

    1 year ago
    Will ping you when I get a chance to circle back to this, thanks!