@Marvin I’ve deploy Airbyte OSS using heml chart to EKS. Curious if I can deploy Prefect jobs via gitlab runner CI job. Currently I’m getting 405 Method Not Ellowed error. Is it because of API key? Is it supported it OSS?

Deploying Prefect jobs via a GitLab runner CI job is supported, but there are some critical points to consider: 1. Authentication: You must use a Prefect API key to authenticate your deployments. This key should be set as an environment variable in your CI/CD pipeline. The exact variable name and how to set it might depend on your GitLab CI configuration. 2. 405 Method Not Allowed Error: This error typically indicates that the HTTP method used in the request is not allowed for the endpoint. Double-check the API call in your CI job to ensure it matches the supported methods. 3. Prefect OSS: The features for CI/CD pipelines using Prefect, including the API key, should work for both Cloud and OSS versions. Ensure you have the correct API key and other deployment configurations. Here’s a sample excerpt from the Prefect docs: URL: https://docs.prefect.io/latest/integrations/prefect-gitlab/

prefect-gitlab library makes it easy to interact with GitLab repositories and credentials. 
Prerequisites: Install prefect-gitlab 
pip install -U prefect-gitlab 
Register newly installed block types:
prefect block register -m prefect_gitlab 
Examples: 
Create blocks with Python code or through the Prefect UI.
Store deployment flow code in a private GitLab repository using GitLabCredentials block.

Make sure your GitLab runner job includes the setup for Prefect and correct environment variables similar to the one provided in GitHub actions documentation like: URL: https://docs.prefect.io/latest/guides/ci-cd/

env:
PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }}

If you continue to face issues, ensure your Prefect worker is correctly subscribed to the work pool and queue. For more detailed troubleshooting, refer to: URL: https://docs.prefect.io/latest/guides/troubleshooting/ If you have specific issues, the Prefect GitHub issues page could also have discussions that might help: URL: https://github.com/PrefectHQ/prefect/issues/405 Feel free to ask if you need further assistance!

What API key to use? @Marvin

For Prefect OSS, you don't use Prefect Cloud API keys since those are specific to Prefect Cloud. Instead, you should create and use an API key configured for your OSS setup. Here’s what you should do: 1. Create an API key: If you haven't created an API key for your Prefect OSS instance, set it up in your configuration or directly if supported by your deployment method. 2. Configuration: Typically, this involves generating a key through your Prefect user interface or command-line if you’ve set up a local Prefect server. Ensure that your

prefect.yaml

or equivalent configuration file includes appropriate authentication settings. 3. Set Environment Variable: In your GitLab CI job, set the environment variable for the Prefect API key. For example:

variables:
  PREFECT_API_KEY: "<your-api-key>"

For detailed guidance, refer to the Prefect documentation on setting up Auth for OSS or contact your deployment admin for specific configurations pertinent to your environment. For more information, visit the official Prefect docs on API keys: URL: https://docs.prefect.io/latest/cloud/users/api-keys/

there is no concept of an API key in open source, this answer is misleading

Ah, right There is no auth. Appreciate it 🙏