does prefect cloud provide gpus??
# prefect-server
n
does prefect cloud provide gpus??
a
No, we don’t have such capabilities yet. But we have partners who provide that. Can you tell us a bit more about your use case?
n
I do image recogniton
so need
gpus
I have done a meeting with richard
he told me that you host on gcp
so can you setup gpus for me with ubuntu?? is it possible I have a standard plan in perfect cloud i have added my credit card also
a
I assume you mean Richard from Coiled? 🙂 The way such partnerships work is that the partner provides the compute infrastructure that can be easily integrated with Prefect. For example, you could set up your cluster with GPUs in your Coiled account and use that in your Prefect flow. Here is an example flow from @Kevin Kho:
Copy code
import coiled
from prefect import task, Flow
import torch
import prefect
from prefect.executors import DaskExecutor
from prefect.storage import GitHub

@task
def abc(x):
    logger = prefect.context.get("logger")
    <http://logger.info|logger.info>(torch.cuda.is_available())
    <http://logger.info|logger.info>(torch.cuda.device_count())
    <http://logger.info|logger.info>(f"Map index {x}")
    return x

with Flow("gpu-test") as flow:
    abc.map([1,2,3,4])

coiled.create_software_environment(
    name="gpu-env4",
    container="gpuci/miniconda-cuda:10.2-runtime-ubuntu18.04",
    conda={
        "channels": ["conda-forge", "defaults", "fastchan"],
        "dependencies": [
            "python==3.8.5",
            "pytorch",
            "torchvision",
            "cudatoolkit=10.2",
            "prefect", 
            "fastai",
            "scikit-image",
            "numpy",
            "dask",
            "bokeh>=0.13.0",
            "dask-cuda",
            "prefect",
            "pandas"
        ]
    })

executor = DaskExecutor(
    cluster_class=coiled.Cluster,
    cluster_kwargs={
        "software": "kvnkho/gpu-env4",
        "n_workers": 2,
        "shutdown_on_close": True,
        "name": "prefect-executor",
        "worker_memory": "4 GiB",
        "worker_gpu": 1,
        "backend_options":{"spot": False}
    },
)

flow.executor = executor
flow.storage = GitHub(repo="kvnkho/demos", path="/prefect/coiled_gpu.py", ref="main")
flow.register("dremio")
So you can see that you can spin up a software environment with GPUs and with all dependencies required by your image recognition code. There are additional options such as using a Kubernetes cluster e.g. AWS EKS with GPUs and you can easily spin up such cluster using eksctl. Finally, with Orion, this will become even easier thanks to annotations.
k
We don’t host any hardware in Prefect so you need to get a GPU from another provider
n
no richard
from perfect
a
I see, our Richard 😄 Can you look into my suggestions from above and check out if one of those options can work for you? Both AWS EKS and Coiled are quite straightforward to set up and if you face issues with any of that, let us know, we are happy to help
n
can you do that for me please I am ready to pay the costs
like you can set it up
a
We don’t do it for privacy reasons. We have a hybrid execution model at Prefect that keeps your code and data secure. Which option do you prefer - Coiled or pure AWS? Let me first provide you some resources about Coiled so that you can see how you can get started with it: • https://docs.coiled.io/user_guide/getting_started.html - this also has a video showing how you can set up your Coiled account •

How To Scale Prefect With Dask Via Coiled

Workflow management with Chris White from Prefect

Workflow automation (Prefect) — Coiled documentationScale your Prefect + Dask Workflows to the Cloud And again, if you have any question along the way you can reach us: • Question about coiled - reach out Coiled support • Question about Prefect’s DaskExecutor - ask here via Slack
n
iam not going to use coiled
at all
only aws
k
Coiled lets you connect your own AWS account
n
no
i dont want coiled at all
a
There are additional options such as using a Kubernetes cluster e.g. AWS EKS with GPUs and you can easily spin up such cluster using eksctl.
n
for that I need AWS account right I need to have gpu quotas right
a
Correct, you need to sign up on AWS. AWS has a service called Sagemaker that makes it easy to build image recognition models and run it on GPUs. Btw, just for your testing purposes, you could start with Google Colab - it’s a free service allowing (limited) access to GPUs but perhaps that’s something you could use to get started, and once you have some models you would like to regularly run in production, you can set up an AWS EKS cluster with GPU and a Prefect KubernetesAgent.
107 Views