hi there. ive just started playing around with prefect and having fun. i have set up various working examples e.g. connecting to static and dynamics dask execution environments.
I am a little confused about how i would combine dask/prefect-server + ui on kubernetes and submit flows from a client (e.g. connecting to a dask scheduler running on kubernetes and being able to monitor them in a UI). The UI could be running anywhere i guess im just not sure how this fits together. prefect-server needs docker compose etc so i guess we dont run this on kubernetes? anyway, a little lost on this monitoring part and not even sure what is the right question. would love to hear folks thoughts on this.
2 years ago
Hi @Saoirse Amarteifio - in short, the first release of Prefect server was really designed to be a locally hosted API (e.g.,
all work occurs on the same machine including the UI).We are working to expose the underlying networking config in a more transparent way, but in the meantime the easiest way to run on kubernetes is to use Prefect Cloud’s API (which is accessible anywhere as long as you have an authorization token). [Note there is a free tier that is suitable for most individual work]
2 years ago
Thanks @Chris White - i also noticed your post in
#prefect-community which links to the medium post for deploying it on GCP/VM which looks good for what I am doing (evaluating how to move our internal airflow jobs onto prefect, respecting some of the kubernetes things we do today).