https://prefect.io logo
Title
a

Anatoly Myachev

03/23/2022, 8:20 PM
Is it possible to configurate Orion to use `KubernetesFlowRunner`+
DaskTaskRunner
that use
dask_cloudprovider.aws.FargateCluster
? There is no logs in Orion UI except
'task-b610b3b0-0' - Crash detected! Execution was interrupted by an unexpected exception.
and
RuntimeError: IOLoop is closed
but it's possible to see it in CloudWatch console. It contains the following error:
Exception: 'RuntimeError("Cannot orchestrate task run \'9987d480-fad1-433a-9749-077a63fbdc0a\'. Failed to connect to API at <http://orion:4200/api/>.")'
. Note: Orion is deployed on Kubernetes via
orion kubernetes-manifest
command.
a

Anna Geller

03/23/2022, 8:22 PM
If you use
KubernetesFlowRunner
, then it makes much more sense to use
KubeCluster
instead of
FargateCluster
? Would you want to give it a try?
this way you can run all components on the same Kubernetes cluster and you can prevent various network issues
are you running this on AWS EKS?
a

Anatoly Myachev

03/23/2022, 8:33 PM
I think it's worth a try with
KubeCluster
. However, I am experimenting with a possible case where there are not enough resources in the current Kubernetes cluster and I want to perform part of the load in AWS cluster (EKS/ECS). So in this case, work will occur with two different clusters. Current K8s cluster is a k3s running on one AWS host.
👍 1
a

Anna Geller

03/23/2022, 8:43 PM
so it's technically not a cluster 😄 You could definitely use EKS on Fargate which would allow you to use all the benefits of Kubernetes but run things on Fargate (getting all the benefits and drawbacks of serverless)
we definitely don't have any ECS Fargate recipes (and no ECSFlowRunner) as of today and I'd say Kubernetes is a way more prevalent use case so if we would create some tutorials on running Dask on a remote cluster, I think we would first focus on Kubernetes use case with
KubeCluster
. If you're interested, you can follow the dask-task-runner tag or the kube-cluster tag on Discourse to get notified about any tutorials/content/recipes about it but of course, if you would want to build and contribute one, that would be super appreciated as well!
👀 1
a

Anatoly Myachev

03/23/2022, 8:51 PM
@Anna Geller thanks, I will take a look
👍 1