https://prefect.io logo
Powered by
Title
c

Chris Goddard

09/10/2020, 7:05 PM
Hey folks - question for you. I'm using prefect cloud to orchestrate my flow runs and running everything within a docker container on GCP. In addition to my more sophisticated flows that I schedule and run using Prefect Cloud, I have some simple flows that run on very frequent increments (every 1-5 minutes). because they're simple and don't really require the monitoring/overhead of prefect cloud and because I didn't want to be always using up a flow slot for them, I just have them running directly as scripts (i.e. calling 
flow.run()
). The problem is, after a period of time, I start getting a persistent error - 
prefect.utilities.exceptions.ClientError: Malformed response received from Cloud - please ensure that you have an API token properly configured
. I'm a little confused because I though if you were just calling 
flow.run()
I wouldn't be interacting with the cloud API at all but clearly because it's in the same environment it's detecting the connection but then perhaps a token is becoming invalid or something? Any ideas?
d

Dylan

09/10/2020, 7:43 PM
Hi @Chris Goddard That’s definitely confusing
Is that error from the process that calls 
flow.run
for your local scripts?
What I suspect this is
With the Prefect CLI, you can call 
prefect login
to login to your Prefect Cloud tenant
The token returned by this method needs to be refreshed every 15 minutes, however
Both your flows running from cloud and your flows running locally are in the same python environment, so when that token expires you could see that error
c

Chris Goddard

09/10/2020, 7:47 PM
got it - I thought that might have something to do with it. how would you recommend keeping that token current? do I need to run 
prefect login
periodically?
d

Dylan

09/10/2020, 7:49 PM
I also would like to make sure that your flows running with 
flow.run
are: - not making a query via a statehandler or a similar mechanism that talks to cloud - not using 
PrefectSecret
s locally by configuring the 
use_local_secrets
option in your 
config.toml
as those options could cause this error
If all of your flows need to communicate with a single tenant, you can set your token in a more long-lived way by grabbing an API token from https://docs.prefect.io/orchestration/ui/team-settings.html#api-tokens and setting
[cloud]
auth_token = {your token}
in your config.toml
c

Chris Goddard

09/10/2020, 7:53 PM
got it. thank you that's really helpful. I think it's being caused by a slack_notifier state handler so that makes sense
d

Dylan

09/10/2020, 7:53 PM
Yup! That would do it
When you initialize the notifier you’ll need to specify 
backend_info=False
and be aware that managing the secret for the notifier can be a little tricky if you have some flows on your instance talking to cloud for secrets and some using local secrets in your 
config.toml
#prefect-communityJoin Slack
Powered by