https://prefect.io logo
Title
a

Azer Rustamov

03/21/2022, 2:05 PM
Hi! How do I mount AWS credentials to
KubernetesFlowRunner
?
a

Anna Geller

03/21/2022, 2:20 PM
Where is your Kubernetes cluster running?
If you are running this on AWS EKS, then the right way of solving that is through IAM roles for service accounts. The easiest way to set this up is using eksctl.
a

Azer Rustamov

03/21/2022, 2:40 PM
I'm trying to run it locally first, then, eventually, will try running on EKS.
a

Anna Geller

03/21/2022, 2:48 PM
To run it on a local Kubernetes cluster in Docker Desktop + Cloud 2.0, this has worked well for me:
from prefect import flow, task, get_run_logger
from prefect.deployments import DeploymentSpec
from prefect.flow_runners import KubernetesFlowRunner
import platform


@task
def say_hi():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello world!")


@task
def print_platform_info():
    logger = get_run_logger()
    <http://logger.info|logger.info>(
        "Platform information: IP = %s, Python = %s, Platform type = %s, OS Version = %s",
        platform.node(),
        platform.python_version(),
        platform.platform(),
        platform.version(),
    )


@flow
def kubernetes_flow():
    hi = say_hi()
    print_platform_info(wait_for=[hi])


DeploymentSpec(
    name="example",
    flow=kubernetes_flow,
    tags=["local"],
    flow_runner=KubernetesFlowRunner(
        env=dict(
            AWS_ACCESS_KEY_ID="xxxxx",
            AWS_SECRET_ACCESS_KEY="yyyyy",
            PREFECT_API_URL="<https://api-beta.prefect.io/api/accounts/account_id/workspaces/workspace_id>",
            PREFECT_API_KEY="pnu_zzzz",
        ),
    ),
)
if you are trying to do the same with Orion services running within the same Kubernetes cluster, you should be able to follow this tutorial and only add the AWS env variables from above.