https://prefect.io logo
Title
a

Assaf Ben Shimon

04/24/2022, 3:24 PM
I'm getting the following error when registering a flow:
Traceback (most recent call last):
  File "/app/biomarkers/extract_markers_image.binary.runfiles/biomarkers/biomarkers/dockers/extract_markers.py", line 2, in <module>
    from biomarkers.batch_processing import orchestrator
  File "/app/biomarkers/extract_markers_image.binary.runfiles/biomarkers/biomarkers/batch_processing/orchestrator.py", line 130, in <module>
    seq_markers_extraction_flow.register("neuralight-research")
  File "/usr/local/lib/python3.9/site-packages/prefect/core/flow.py", line 1708, in register
    registered_flow = client.register(
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 839, in register
    project = self.graphql(query_project).data.project  # type: ignore
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 452, in graphql
    result = <http://self.post|self.post>(
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 407, in post
    response = self._request(
  File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 649, in _request
    raise AuthorizationError(
prefect.exceptions.AuthorizationError: Malformed response received from Cloud - please ensure that you are authenticated. See `prefect auth login --help`.
Any idea what can cause it? I've already authenticated using my API key
a

Anna Geller

04/24/2022, 3:28 PM
It looks like you have some custom registration logic and the environment from which you are registering the flow isn't authenticated to Prefect Cloud Can you share your registration logic from
orchestrator.py
? Also, if you are just getting started with Prefect, you may want to start with Prefect 2.0 directly
a

Assaf Ben Shimon

04/24/2022, 3:32 PM
The registration logic is simply
flow_name.register("project-name")
, do I need to do anything else? (also, I run the code from docker, if that changes anything)
a

Anna Geller

04/24/2022, 4:00 PM
You just need to make sure your registration environment is authenticated with Prefect Cloud. Perhaps you can add this environment variable to your registration Docker container and try again?
export PREFECT__CLOUD__API_KEY="your_key"
a

Assaf Ben Shimon

04/24/2022, 4:02 PM
Thanks! And what if I want to run locally (with prefect server?) Is the registration process the same?
Sorry for the trouble, I'm new to perfect and afraid I did something wrong. I want to execute flows locally and track their progress using prefect server (let's take cloud out of the picture for now). I have a docker image which creates the flows (if I simply run it, the flows are executed as expected; so what's left is just orchestrating it) I've tried running a local agent from the CLI, but that doesn't seem to be enough as I need to somehow register my flows so that it'll track them, right? Also, when I run
prefect server start
a local server starts and I can see the UI, but nothing updates there and I can't see my flows. So what's the right way to glue these pieces together? Thanks a lot in advance!!
a

Anna Geller

04/24/2022, 4:41 PM
You're very welcome! Are you already in some way committed to Prefect Server? If you install and start Prefect 2.0 instead, you don't have to register your flows - simply running your flows locally will allow you to track them in the UI
I've tried running a local agent from the CLI, but that doesn't seem to be enough as I need to somehow register my flows so that it'll track them, right?
Yup exactly - you start an agent with e.g. label "dev" and you attach the same label to your
flow.run_config
and when you then register and run your flow, it will match it with the agent via a label. This page explains it more for 1.0 you can also check this Discourse topic with links to resources that will help understand various concepts
a

Assaf Ben Shimon

04/24/2022, 8:23 PM
Thanks Anna!! Server is running, agent is up, flows are executing - but I can't see anything on the server's UI. When looking at the console, I see this: Any idea what's the reason? (I'm running on an EC2 instance without any inbound or outbound traffic filtering, and use port forwarding to access the server from my machine)
Update: I've also tried adding the 'expose' flag (
prefect server start --expose
), but no change 😕
k

Kevin Kho

04/24/2022, 9:04 PM
Did you set something like this
[server]
  
  [server.ui]
    
    apollo_url = "<http://YOUR_MACHINES_PUBLIC_IP:4200/graphql>"
in the
config.toml
before doing the
prefect server start --expose
? Looks really weird why it’s null
a

Assaf Ben Shimon

04/24/2022, 9:59 PM
Should I have? (Currently there is no file at
~/.prefect/config.toml
)
a

Anna Geller

04/24/2022, 10:21 PM
So you spun up EC2, started Prefect Server and a local agent there, and now you try to connect to the UI from your local machine, correct? The problem now is to ensure that you also follow all those steps on your local machine (from which you try to access the UI): • install prefect, • switch backend to
server
, • adjust (or create if not available) your
~/.prefect/config.toml
and then try accessing the UI if you still have trouble, check those Discourse topics: • https://discourse.prefect.io/t/how-to-deploy-self-hosted-server-and-register-first-flows-to-the-server-backend/85https://discourse.prefect.io/t/can-t-connect-to-a-prefect-server-from-other-machine-despite-port-4200-being-open/128