Igor Morgunov
01/16/2023, 11:56 AMgithub_block = GitHub.load("XXXXXX")
cluster_config_block = KubernetesClusterConfig.load("data-engineering")
k8s = KubernetesJob(
cluster_config=cluster_config_block
)
deployment = Deployment.build_from_flow(
flow=test,
name="test",
version="0.0.1",
tags={"from-python-object"},
parameters={"name": "test-run"},
infra_overrides={"env": {"PREFECT_LOGGING_LEVEL": "DEBUG"}},
work_queue_name="test",
infrastructure=k8s,
storage=github_block,
)
if __name__ == "__main__":
result = deployment.apply()
Anna Geller
01/16/2023, 6:53 PMIgor Morgunov
01/17/2023, 10:18 AMSubmission failed. kubernetes.client.exceptions.ApiException: (401) Reason: Unauthorized HTTP
Anna Geller
01/17/2023, 12:17 PMeksctl
to create a minimal K8s cluster e.g. single node
• use prefect kubernetes manifest agent
, adapt if needed e.g. namespace and apply this deployment
• then you have the agent - all is left is K8s job block and a deployment