https://prefect.io
Join Slack
Hi all! I'm looking to leverage Prefect Parameters in order to sometimes manually enter run paramete...
# ask-community
c
Hi all! I'm looking to leverage Prefect Parameters in order to sometimes manually enter run parameters if a pipeline needs to be re-run. I'm looking to clarify what exactly the difference between Context and Parameters is and how to use them correctly as I'm a little confused by the documentation.
k
Hey @Charles Liu, I think if you are passing in data, it should be in the form of Parameters. The context is more meant to hold these things that set at runtime (flow name, today, task id). The parameters are under the context. Context is more of configurations around the flow. You could use it though for stuff like environments. Like this from the docs.
Copy code
environment = "prod"
user = "${environments.${environment}.user}"

[environments]

    [environments.dev]
        user = "test"

    [environments.prod]
        user = "admin"
If you have this in the config.toml, you can then check the value inside your Flow. So think of these as settings you want on the agent also, whereas parameters are really about the Flow itself agnostic to where it is running. There is some overlap in what can be achieved with parameters an the context though.
upvote 1
c
Okay that does clarify things! Definitely not trying to use Context in this instance then.
What I'm actually trying to accomplish is override an integer. I know there's the --parameter-string for CLI, but I also noticed there's a Parameters window in Settings on the flow page on the front end. Is there something I need to do to activate that window? Currently it says my flow has no default Parameters but also does not give me an empty json window like in the docs.
k
Yeah I think you need to register your flow with Parameters
Copy code
x = Parameter("x", default=1)
Then you should see that window populated. And then you can specify the default there and override this. But if you just want one run with a different Parameter value, you should click the 
Run
instead of 
Quick Run
and it will let you set the Parameters there for that individual run.
c
okay so is this the right way to use it for example:
Copy code
important_number = Parameter("important_number", default=3)

@task()
def this_task():
   task_params:{
       "important_number": 3
        }

with Flow()...etc.
k
Copy code
@task
def abc(x):
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(x)
    return x

with Flow("context-test") as flow:
    x = Parameter("x", default=3)
    a = abc(x)
c
ah okay it has to go in the flow block gotcha
k
The task will just treat it as a regular input. The Parameter is a special task.
c
ahhhh okay that makes it make a little more sense when you put it that way
Say I want to use timedelta along with Parameters, is there a way to access the Parameters object in the flow block?
k
What do you mean by access? Do you mean something else other than passing it to another task?
c
Hmm you bring up a good point I could just operate on the value within a task and pass out the result. Basically the integer I want to edit helps create another value; and that value I want to set as the default of another Parameter so that it defaults to an updated value every time.
Although having that logic mushed into another task or having its own task seems unwieldily for such a small step? Unless that's best practice to taskify every little thing.
k
Oh you probably need a task for that. Yes most things in the Flow block need to be a task but you can make a lambda to make it faster. 
task(lambda x: x+1)(value)
🙌 1
c
!!! will give that a shot
k
I realized I don’t think you can do that where you do:
Copy code
with Flow("context-test") as flow:
    x = Parameter("x", default=1)
    y = Parameter("y", task(lambda x: x + 1))
    a = abc(y)
The default needs to be static because it’s registered and saved during build time. 
y
here shouldn’t be a Parameter. You can just do the 
y = task(lambda x: x +1)(x)
âś… 1
c
Understood, will let you know if it works out
Okay so followup question: how do you unregister a parameter?
I removed one from my flow block and it's still there on the front end
k
Even when you re-register the flow? Did you try checking the latest version?
c
I believe the version number is incrementing but the removed Parameter() is still on the front end Run page.
The way I'm re-registering my flow is running the .py file
k
Hey I’ll test this now
You mean here right?
It is working for me (I re-registered with less parameters)…could you try it on a separate flow and see if it still lingers in the UI?
c
Will ping you when I get a chance to circle back to this, thanks!
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by