https://prefect.io logo
KeyError with environment variable
s

Samuel Kohlleffel

12/07/2022, 6:29 PM
Hello, it appears that creating a flow deployment fails when I set an env var as the default value for a task input parameter. I assume that's because the environment that creates the flow deployment (GitHub Actions Runner) doesn't have that env var set. Is this a bug? I feel like I should be able to create and apply the deployment with no issues. Examples in thread.
1
Deployment works:
Deployment fails:
p

Peyton Runyan

12/08/2022, 10:54 AM
Is there a stack trace that came with the error?
And when you say deployment fails, do you mean the build step fails? Or the deployment fails when you run it? If I've got more details on the error I should be able to get a better feel for it
Ah - I'm actually in the middle of a PR that will fix the
ScriptError
messages so that you get a more useful error message. Just reproduced this. You get a
KeyError
. It runs just fine if the environment variable is present
If you want to get an environment variable and not throw a
KeyError
when it's not present, you need to use
os.getenv(<MY_VAR>)
which functions the same ways a
dict.get()
. This applies to all of python, not just prefect.
s

Samuel Kohlleffel

12/08/2022, 12:42 PM
Ah, OK. Will do. Thanks for the help!
👍 1
@Peyton Runyan Is there a way to ensure
Deployment.build_from_flow()
throws a Python exception when any error occurs. Right now, we just get a log message showing the generic error, but the script continues to run as if no error occurred.
p

Peyton Runyan

12/13/2022, 10:54 PM
Can you past the whole error trace?
s

Samuel Kohlleffel

12/14/2022, 1:48 PM
Here's the error log. Unfortunately, a trace isn't logged out when an error occurs during
Deployment.build_from_flow
.
13:46:24.407 | ERROR   | root - Script at '/workspaces/nimbus/orchestration/flows/hvr_alerting/hvr_alerting.py' encountered an exception
For context, we deploy our flows during a CI job (GitHub Actions). Because the script that builds the flow deployments and pushes them to Prefect Cloud doesn't throw an error, we could in theory have a failed deployment, but our GH Action won't fail and we won't know about it. Prefect won't update/create that deployment with the error, which is an issue - especially if we don't know about it.
p

Peyton Runyan

12/14/2022, 2:01 PM
I'm gonna need a hand reproducing this. Can you get a minimum example locally that results in this same situation that I can copy and try out?
s

Samuel Kohlleffel

12/14/2022, 2:29 PM
So it turns out it wasn't an issue on y'alls end. I just realized we're handling errors of
load_flow_from_script
incorrectly. Sorry to waste your time. I really appreciate the help though. I would've never found the issue had you not requested a working example.
❤️ 1
p

Peyton Runyan

12/14/2022, 2:29 PM
No worries man!
Glad you got it figured out 🙂
💯 1