Andrew

    Andrew

    3 months ago
    Question for anyone who can help me with a scalable installation of the open-source edition. After deploying 1.2.2 (ui/core/agents) with helm to kubernetes, I'm not able to register flows to the cluster due to an API auth error. Given that generating API keys through the UI are not supported on self-installs (greyed out), is there a default key or another way to do this via cli? Can it be disabled? Surely there are ways to handle this among other basic things, such as passing secrets, without being required to use an official paid version of the Cloud UI? Thanks
    Anna Geller

    Anna Geller

    3 months ago
    Are you just getting started with Prefect? If so, perhaps it's better for your use case to start with 2.0? Check this to decide Making Prefect Server scalable is certainly possible but not easy - check this for more info API Keys are for Prefect Cloud, not for Server - if you still want to deploy Server with Helm, check this for more resources - the API Key can be left as an empty string for Server and when switching backend to Server you won't need any API key
    prefect backend server
    Andrew

    Andrew

    3 months ago
    Thanks Anna. This is a very similar use case to mine. I need to run thousands of concurrent tasks every few seconds across a powerful on-premise k8s cluster for a regulated financial trading firm where vendor cloud services, even if only sharing metadata, will never be an option for us. I've just discovered Prefect yesterday but so far it's been straight forward. I'll try to spin up a 2.0 (server) instance and simulate our workload. We have various python and environment dependencies, such as custom python libraries and secured microservice APIs that Prefect tasks will need to interact with in real time. From what I've read, this is all doable. Will post an update here shortly.
    Jenia Varavva

    Jenia Varavva

    3 months ago
    @Andrew to make the OS server work for me I had to (a) create the “default” tenant (either a job included in the helm chart, or
    PREFECT__CLOUD__API=http://<your prefect apollo server address>/graphql  prefect server  create-tenant  -n default
    From there the default python client wouldn’t just work for me, but I had to:
    client = Client(prefect_api_server)
    tenants = client.get_available_tenants()
    client.tenant_id = tenants[0]['id']
    Otherwise it would complain about the API key
    Andrew

    Andrew

    3 months ago
    Update - the issue was that I hadn't issued "prefect backend server" on the client. The cluster setup was fine. As for using Prefect 2.0, I don't see a docker image supporting use of the Orion UI. It seems like the documentation is omitting certain obvious steps regarding self-hosting for cloud experts and it's frustrating. A simple one-pager for deploying Prefect 2.0 via helm would make this way easier and probably lead to more interest in using Prefect.
    Anna Geller

    Anna Geller

    3 months ago
    Andrew, check out this list of resources for self-hosting 2.0 We do have a Docker image on Dockerhub that contains everything, incl. the UI e.g.
    docker pull prefecthq/prefect:2.0b5-python3.9