Question for anyone who can help me with a scalabl...
# prefect-community
a
Question for anyone who can help me with a scalable installation of the open-source edition. After deploying 1.2.2 (ui/core/agents) with helm to kubernetes, I'm not able to register flows to the cluster due to an API auth error. Given that generating API keys through the UI are not supported on self-installs (greyed out), is there a default key or another way to do this via cli? Can it be disabled? Surely there are ways to handle this among other basic things, such as passing secrets, without being required to use an official paid version of the Cloud UI? Thanks
a
Are you just getting started with Prefect? If so, perhaps it's better for your use case to start with 2.0? Check this to decide Making Prefect Server scalable is certainly possible but not easy - check this for more info API Keys are for Prefect Cloud, not for Server - if you still want to deploy Server with Helm, check this for more resources - the API Key can be left as an empty string for Server and when switching backend to Server you won't need any API key
Copy code
prefect backend server
a
Thanks Anna. This is a very similar use case to mine. I need to run thousands of concurrent tasks every few seconds across a powerful on-premise k8s cluster for a regulated financial trading firm where vendor cloud services, even if only sharing metadata, will never be an option for us. I've just discovered Prefect yesterday but so far it's been straight forward. I'll try to spin up a 2.0 (server) instance and simulate our workload. We have various python and environment dependencies, such as custom python libraries and secured microservice APIs that Prefect tasks will need to interact with in real time. From what I've read, this is all doable. Will post an update here shortly.
j
@Andrew to make the OS server work for me I had to (a) create the “default” tenant (either a job included in the helm chart, or
PREFECT__CLOUD__API=http://<your prefect apollo server address>/graphql  prefect server  create-tenant  -n default
From there the default python client wouldn’t just work for me, but I had to:
Copy code
client = Client(prefect_api_server)
tenants = client.get_available_tenants()
client.tenant_id = tenants[0]['id']
Otherwise it would complain about the API key
a
Update - the issue was that I hadn't issued "prefect backend server" on the client. The cluster setup was fine. As for using Prefect 2.0, I don't see a docker image supporting use of the Orion UI. It seems like the documentation is omitting certain obvious steps regarding self-hosting for cloud experts and it's frustrating. A simple one-pager for deploying Prefect 2.0 via helm would make this way easier and probably lead to more interest in using Prefect.
a
Andrew, check out this list of resources for self-hosting 2.0 We do have a Docker image on Dockerhub that contains everything, incl. the UI e.g.
Copy code
docker pull prefecthq/prefect:2.0b5-python3.9