https://prefect.io logo
i

Igor Yamamoto

08/02/2023, 12:48 PM
Hi guys!! I recently started playing around with Prefect, and I am currently trying to host my own server. So far, I've deployed the Helm Charts for the Prefect Server and Prefect Worker on a Kubernetes cluster. I would like to understand how can I submit my flows, located on my local machine, to my remote Prefect server. As from this discussion and from this tutorial, I should set the backend to
server
mode. But when I run the
prefect backend server
command, I get the following error. My prefect version is 2.10.4. Am I doing something wrong?
d

Deceivious

08/02/2023, 12:51 PM
how can I submit my flows
This can be interpreted differently. 1. If you want the prefect server to trigger your flows , look into deployments. 2. If you want the flows you run locally to be registered on prefect server [and displayed on the UI] , Look into prefect profile.
i

Igor Yamamoto

08/02/2023, 12:56 PM
From what I understood, a flow deployment should submit my flow to be run on the remote workers I setr up along with the Prefect Server, right? I can't clearly understand why any setup regarding profiling would change the behaviour in which my flows run.
d

Deceivious

08/02/2023, 1:01 PM
Unsure what you mean. But a deployment just tells the infrastructure [kubernetes agent job in your case] to execute a code in specified location at given times. The server doesnt need to have access to the code . The workers on the other had should have access to the code to be executed.
Also both the links you are following are old. The stack overflow is actually prefect 1 and not prefect 2
👍 1
i

Igor Yamamoto

08/02/2023, 1:05 PM
Ok, but what exactly do you mean by "having access to the code"? How should I grant access to my code? From what I understood, the steps would be to: 1. Deploy a Prefect Server 2. Create a Work Pool 3. Deploy a Prefect Worker 4. Attach the worker to the pool 5. Submit my local flow to the server The fifth step is not clear to me as how I should do it. Am i getting it wrong?
d

Deceivious

08/02/2023, 1:07 PM
You have the right steps. Now you just need to deploy your flows. Look into deployment.
i

Igor Yamamoto

08/02/2023, 1:24 PM
Good to know that the conceptual steps are clear. But, considering that I am using a self-hosted Prefect Server, how exactly should I overwrite the default setting (that is Prefect Cloud) and use instead my own remote server? I saw that there is a
deployment.yaml
file that I can use to pre-set some parameters regarding the deployment process. Should I edit this file to point to my target server IP?
d

Deceivious

08/02/2023, 1:26 PM
Multiple approaches to this. But you can configure your local profile to connect to your server IP https://docs.prefect.io/latest/concepts/settings/
disclaimer: I dont use deployment.yaml so i have no idea.