https://prefect.io
Join Slack
Hello! In Prefect 1.0, I have my flow run_config set to call a function set_run_config. From that fu...
# ask-community
j
Hello! In Prefect 1.0, I have my flow run_config set to call a function set_run_config. From that function, is it possible to access the labels that have been added to the flowrun?
✅ 1
a
why would you want to retrieve run config labels?
it's not possible but it also not needed - labels are really a build-time concept not relevant for the flow code
curious about your use case for this - is it to manage environments like dev/prod distinction? if so, check out https://discourse.prefect.io/t/how-to-distinguish-between-various-environments-development-staging-production-in-prefect-flows/179
and in Prefect 2.0, this becomes much easier thanks to workspaces and profiles + improved CLI-driven deployment UX - no need to hardcode environment information in your code thanks to the 2.0 awesomeness 🎉
j
Thanks! Yes, that’s the use case, to manage environments. I’ve been looking at that post, but can’t see how you’d extend it to add a third environment, say staging. The code there seems to deal with two environments, dev (running as local), or prod (running on k8s). I’m mulling a move to 2.0 (and love the new features/ways of working!), but since we’ve invested quite a bit in ECS and 2.0 doesn’t support that yet, I’m a little reluctant
a
soon 🙂
j
🙂
Good to hear!
Am I right in thinking that code only allows you to deal with two environments? Worried that I’m missing something…then I’ll stop fiddling around trying to get something to work!
a
as many environments as you like
j
Really sorry, I’m being dumb here, but I can’t quite figure it out! 🙈 Just wondering how I’d add another environment to the below example from your Discourse post:
Copy code
def set_run_config(local: bool = False) -> RunConfig:
    if local:
        return LocalRun(labels=["dev"])
    aws_account_id = Secret("AWS_ACCOUNT_ID").get()
    return KubernetesRun(
        labels=["prod"],
        image=f"{aws_account_id}.<http://dkr.ecr.us-east-1.amazonaws.com/prefect-dbt-k8s-snowflake:latest|dkr.ecr.us-east-1.amazonaws.com/prefect-dbt-k8s-snowflake:latest>",
        image_pull_policy="IfNotPresent",
    )
There’s the 
if local
line and then it falls back to 
KubernetesRun
. Not sure how I’d add another one there?
a
Copy code
def set_run_config(env: str = "local") -> RunConfig:
    if env == "local":
        return LocalRun(labels=["dev"])
    elif env == "staging":
        return DockerRun(labels=["staging"])
    aws_account_id = Secret("AWS_ACCOUNT_ID").get()
    return KubernetesRun(
        labels=["prod"],
        image=f"{aws_account_id}.<http://dkr.ecr.us-east-1.amazonaws.com/prefect-dbt-k8s-snowflake:latest|dkr.ecr.us-east-1.amazonaws.com/prefect-dbt-k8s-snowflake:latest>",
        image_pull_policy="IfNotPresent",
    )
just example
j
Awesome! Thanks! And one final question, how do I set 
env
? Or how will 
env
be different in each environment?
a
This is just a string passed to the function, you could call it Bratwurst if you want to :)
j
Ah ok, I suppose my question was more how to set the value in the flow, so for example this works:
Copy code
with Flow(FLOW_NAME, storage=set_storage(FLOW_NAME), run_config=set_run_config(env="test"),) as flow:
But this doesn’t:
Copy code
env = os.getenv('ENV_SWITCH')
with Flow(FLOW_NAME, storage=set_storage(FLOW_NAME), run_config=set_run_config(env="test"),) as flow:
I need to be able to have it set dynamically for this use-case
a
It doesn't work that way, this value must be sent as metadata to the API -- I think what you need is a different cloud tenant to manage multiple environments but given that Cloud 2 and Prefect 2 is the LTS product, I wouldn't optimize for that now. In 2.0 switching between environments is as simple as switching your CLI profile
j
Which API are you referring to? Can I pass to the graphql api?
So normally if you’re using the proposed solution here, you’d need to register a different flow for each environment (staging, test and prod)?
6 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by