https://prefect.io logo
Channels
Powered by
a

Assaf Ben Shimon

05/09/2022, 11:29 AM
Hi all, I have an authentication problem:
Copy code
prefect.exceptions.AuthorizationError: Malformed response received from Cloud - please ensure that you are authenticated. See `prefect auth login --help`.
However when I run 
prefect auth status
it says I'm connected and authenticated. Any idea what's wrong?
a

Anna Geller

05/09/2022, 11:39 AM
Under what circumstances do you get that error? it could be that your agent/execution layer is not authenticated
Can you also confirm that you are on Prefect Cloud? Usually, I see this kind of error when Prefect Server users are hitting Cloud endpoints or didn't switch backend to server as here
a

Assaf Ben Shimon

05/09/2022, 11:49 AM
The agent is also authenticated and I can see it in cloud.prefect.io. And my backend is cloud.
a

Anna Geller

05/09/2022, 11:53 AM
1. Can you share the output of 
prefect diagnostics
in both your flow registration environment and your agent? 2. What agent do you use? 3. How did you start your agent? Does your agent has the 
backend
switched to 
cloud
? 4. Can you share part of your failing flow code that shows the storage and run config? In the worst case, you could recreate the API key and try logging out and in again with the new key on both your agent and your registration environment, since the API key may have expired
Regarding Orion - if you are just getting started with Prefect, it may actually be beneficial to start directly with Prefect 2.0 and Cloud 2.0 - check this post for more info
a

Assaf Ben Shimon

05/09/2022, 11:56 AM
mmm, might the problem be that I'm running my flows from a docker?
a

Anna Geller

05/09/2022, 12:03 PM
Can you share more about how you do it?
a

Assaf Ben Shimon

05/09/2022, 12:04 PM
I have a bazel 
py3_image
rule that I run (
bazel run image_name
)
so basically building and starting a docker that executes the flows
a

Anna Geller

05/09/2022, 12:10 PM
can you share a bit more? this doesn't look like a "golden path" so I'd need more info 😄 usually you would need to register your flow, Prefect then takes care of the rest could you please answer all my questions to approach it step-by-step?
a

Assaf Ben Shimon

05/09/2022, 12:11 PM
Copy code
{
  "config_overrides": {
    "cloud": {
      "api_key": true
    }
  },
  "env_vars": [],
  "system_information": {
    "platform": "Linux-5.13.0-1022-aws-x86_64-with-glibc2.29",
    "prefect_backend": "cloud",
    "prefect_version": "1.1.0",
    "python_version": "3.8.10"
  }
}
Using local agent
Started with CLI: 
prefect agent local start
after adding the API key to the config file
And this is how I registered: (replaced actual names with placeholders)
Copy code
for path in paths:
    my_flow.register("my-project-name")
    my_flow.run(path = path, version = version)
a

Anna Geller

05/09/2022, 12:15 PM
with a local agent, you need to be particularly careful to ensure matching labels - ideally, you can try reregistering your flow from the same server from which you run your flow - otherwise, depending on your storage config, the flow code may not be found
a

Assaf Ben Shimon

05/09/2022, 12:23 PM
Isn't that what I did? I called 
register
and then 
run
a

Anna Geller

05/09/2022, 12:24 PM
#1 Why do you run your flow during registration - is this intended? #2 I'm not aware of path and version on the flow.run method - are those your flow parameters? if so, usually you would need to put those into your 
parameters
dict
Copy code
def run(
        self,
        parameters: Dict[str, Any] = None,
        run_on_schedule: bool = None,
        runner_cls: type = None,
        **kwargs: Any,
    ) -> Union["prefect.engine.state.State", None]:
also note that flow.run() will just run your flows locally, it doesn't run on the cloud backend so Auth to Cloud wouldn't matter in that case
perhaps as a solution, you could look at Prefect CLI? running and registering flows via CLI is much less confusing and you don't need any extra registration scripts to bulk-register flows, simply doing:
Copy code
prefect register --project xyz -p flows/
will register all flows from the dir flows/
a

Assaf Ben Shimon

05/10/2022, 7:36 AM
1. Yes, but maybe not the right pattern., Also, note that I have a flow that I call many times with different parameters. 2. So what's the right way to run a flow in the cloud? Must I initiate the run from the cloud UI?
a

Anna Geller

05/10/2022, 12:42 PM
given that you asked how to run parametrized flows, check this post that discusses this in detail
188 Views
Powered by