https://prefect.io logo
y

Yongchan Hong

02/13/2022, 1:38 PM
Hi I am totally new to Prefect and trying to figure things out. I have successfully deployed Prefect Server using helm with my own ingress host. Now I am trying to register flow in my own notebook and wondering how to connect to my own server. Can someone guide me?
k

Kevin Kho

02/14/2022, 2:31 PM
Hi @Yongchan Hong, you need you set your backend to server with
prefect backend server
and also in your
config.toml
file, point to the server address with
Copy code
[server]
endpoint = "YOUR_MACHINES_PUBLIC_IP:4200/graphql"
y

Yongchan Hong

02/14/2022, 3:46 PM
@Kevin Kho Hi Kevin! So I am trying to use my own Jupyter Hub cluster, and wondering where to find config.toml inside my Jupyter Hub cluster. Can you guide me through?
k

Kevin Kho

02/14/2022, 3:59 PM
You don’t necessarily need it as a
config.toml
. You can have an environment variable
PREFECT__SERVER___ENDPOINT
also and it will do the same thing
y

Yongchan Hong

02/14/2022, 4:38 PM
@Kevin Kho Why is it telling me to set API key for authentication? Here is my code:
Copy code
!prefect backend server
import os
os.environ['PREFECT_SERVER__ENDPOINT'] = "<https://prefect-apollo-dev.xtrm-data.io/graphql>"
from prefect import Client
client = Client()
client.create_project(project_name="Hello, World!")
@Kevin Kho Let me know if my approach is wrong!
k

Kevin Kho

02/14/2022, 4:44 PM
Is this from a new kernel? Because when
prefect
is imported, it loads in the config (including the env variables). So the environment variable needs to be set before Prefect is imported. Your order is correct, but if it’s asking for an API key, it looks like it didn’t take effect
y

Yongchan Hong

02/14/2022, 4:52 PM
Thanks for the support. Do you mind if I can DM you? Or is talking through thread more convenient? Feels like my understanding about prefect is a bit low. Now it shows me connection refused error with:
Copy code
HTTPConnectionPool(host='localhost', port=4200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f070a1b2510>: Failed to establish a new connection: [Errno 111] Connection refused'))
@Kevin Kho
k

Kevin Kho

02/14/2022, 4:58 PM
Uhh the thread is fine I will see it or sometimes my colleagues will jump in too. This looks like you did not set the address of the server when trying to connect because it’s still
localhost
right? You need to point to your server with your
config.toml
file.
Copy code
[server]
endpoint = "YOUR_MACHINES_PUBLIC_IP:4200/graphql"
y

Yongchan Hong

02/14/2022, 5:00 PM
Yeah I have deployed Prefect Helm with my own ingress. Now I am trying to connect outside, which is from jupyterhub, and I set
PREFECT__SERVER___ENDPOINT
as you told me… but it is still not working. @Kevin Kho Am I understanding something differently?
k

Kevin Kho

02/14/2022, 5:02 PM
Oops sorry I mistook this for another similar thread going on. Yeah it looks like the env variable is not taking effect so it’s probably better we do it through the
config.toml
. I don’t know where this is on JupyterHub though. Give me a second to look
y

Yongchan Hong

02/14/2022, 5:04 PM
Thanks 🙏 Hope I an do something with config.toml since I am JupyterHub and cant find one
k

Kevin Kho

02/14/2022, 5:04 PM
Do you have a
~/.prefect
directory?
If there is no
config.toml
, you can just create one. You can just navigate in the file explorer right?
y

Yongchan Hong

02/14/2022, 5:12 PM
You saved me! It works after manually creating config.toml. Thank you so much. @Kevin Kho
k

Kevin Kho

02/14/2022, 5:14 PM
Nice!
y

Yongchan Hong

02/14/2022, 5:14 PM
So quick question about KubernetesRun - should i specify service_account_name and image when I want to use Kubernetes Agent?
k

Kevin Kho

02/14/2022, 5:17 PM
Yes otherwise it just uses the default (default Prefect image). The image you specify just needs to be in a place the agent can pull it from
y

Yongchan Hong

02/14/2022, 5:44 PM
Okay this may sound dumb - but task is pending infinitely…So I register project and try to run flow.. from prefect import task, Flow, Parameter from prefect.run_configs import KubernetesRun
Copy code
run_config = KubernetesRun(
        labels=["dev"],
        service_account_name="prefect-server-serviceaccount"
    )

@task(log_stdout=True)
def say_hello(name):
    print("Hello, {}!".format(name))


with Flow("My Second Flow", run_config=run_config) as flow:
    name = Parameter('name')
    say_hello(name)


flow.register(project_name="Hello, World!")

client.create_flow_run(flow_id='5bcaf7c8-e8ac-4b4e-9f05-527dd610a12a', parameters={'name':'world'})
But it seems like it create flow run but it never actually runs at all… Is my agent configuration wrong? Also, when just run flow like flow.run(name=‘world’), is it running inside local agent? Sorry for repeated question 😞 @Kevin Kho
k

Kevin Kho

02/14/2022, 5:46 PM
This sounds that the labels for the agent and runconfig do not match. Agents can only pick up flows with matching labels. What are the labels for your agent here?
y

Yongchan Hong

02/14/2022, 6:03 PM
Hmm I found out that label called ‘jupyter-xxx’ is inserted when I am registering my flow… will try to figure out
k

Kevin Kho

02/14/2022, 6:08 PM
You can add it to the storage.
flow.storage = Local(…, add_default_labels=False)
y

Yongchan Hong

02/14/2022, 6:40 PM
@Kevin Kho okay this maybe one last question. Now it founds agent, and it was successfully submitted. But it is infinitely stuck in submitted state. Any ideas..?
k

Kevin Kho

02/14/2022, 7:04 PM
On Kubernetes, I think this means that it can’t get the hardware it needs to create the Flow
Your cluster should have pods in
Not Ready
state with some rror message
r

Ricardo Gaspar

02/16/2022, 6:26 PM
hey @Kevin Kho; I’ve run into the same issue. The docs were confusing on how to setup your local prefect cli to point to a Prefect Core Server instance. Fortunately I’ve found this thread here. Do you think the docs are out of date? • It mentions
[api]
- https://docs.prefect.io/core/concepts/configuration.html#extensions • It’s not clear if it’s something that a user needs to do on its prefect CLI installation or something to be done as part of server deployment - https://docs.prefect.io/orchestration/faq/config.html#connecting-to-a-different-api-endpoint
k

Kevin Kho

02/16/2022, 6:32 PM
I don’t think it’s out of date because you can set HOST and PORT and it will combine to form ENDPOINT here. That API is on the server start up side to determine where to host the API
r

Ricardo Gaspar

02/16/2022, 6:32 PM
I’ve managed to set my configuration using the following
config.toml
file:
Copy code
[server]
host_ip = "X.Y.Z.W"
host = "http://${server.host_ip}"
host_port = "4200"
endpoint = "${server.host}:${server.port}"
k

Kevin Kho

02/16/2022, 6:32 PM
API link
r

Ricardo Gaspar

02/16/2022, 6:36 PM
that api config with host and url is not working for me though. But the server config is. I’m running prefect CLI version
0.15.13
Copy code
prefect get flows                                                                                                                                                                                                 <aws:sdlc-admin>
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
k

Kevin Kho

02/16/2022, 6:39 PM
Did you use helm or the
prefect server start
?
r

Ricardo Gaspar

02/16/2022, 6:40 PM
the prefect server start
I’m using my mac to connect to a server deployed on an AWS EC2 instance; using the
prefect server start
. I’ve followed this guide: https://pasaentuciudad.com.mx/deploying-prefect-server-with-aws-ecs-and-docker-storage/
k

Kevin Kho

02/16/2022, 6:42 PM
A couple of questions: 1. Are you trying to connect from a remote machine? If you are, you need to start with the
--expose
flag.
prefect server start --expose
2. Was starting successful? Can you go check if the GraphQL API is up on server?
There is a note on the
--expose
flag here
r

Ricardo Gaspar

02/17/2022, 4:13 PM
yes the remote machine started the prefect server with the
--expose
flag. with my config.toml file on my mac I’m able to interact with the prefect server and register flows. I’m also able to use the graphql API to trigger flow runs
k

Kevin Kho

02/17/2022, 4:14 PM
Nice! You good now? or did you mean you already did that but it didnt work?
r

Ricardo Gaspar

02/17/2022, 8:18 PM
yes. the way I did, with the config.toml file I’ve sent is working for me now. thank you for all the help 🙂
y

Yongchan Hong

02/18/2022, 4:23 AM
Thank you so much!
6 Views