Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hello,
In Prefect1 I can specified asked resource with `KubernetesRun`  (`cpu_request`, `memory_request` ,`cpu_limit`, `memory_limit`)
In Prefect2 I use `KubernetesJob` , how can I specify the same parameters?

HI Aleksandr,

You can create a job template and add them to the spec, or modify them using customization parameters via json patch

<https://discourse.prefect.io/t/creating-and-deploying-a-custom-kubernetes-infrastructure-block/1531>

<@U03FKQX1TH9> My flow contains several parallel tasks. But when I made the flow runs in k8s job infrastructure block, I see all tasks runs in sequential manner. Will changing “.sepc.parallelism” value in job template could make Prefect engine run tasks in parallel fashion? What is the execution model? One parallel task per PVM (Python VM process) inside the same pod or one parallel task per pod?

Each “flow” is a k8s job, which is a pod. If you’re looking for individual tasks to be spawned as jobs themselves, you’d probably want to be looking at Dask

So in a nutshell, there are no parallelism in k8s job for a flow with concurrent tasks. Is that correct?

Thanks. We had a hard time to figure out how to hook up an existing Dask cluster from Prefect 2. I saw an example to specify Dask task runner in the flow script. But this doesn’t sound like the rest deployment execution model. There is no infrastructure block for that. It is strange.

<@U03FKQX1TH9> do you know if running tasks as independent pods (kubernetes-job) is something that is on the pipeline to be developed?