Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Is it possible to run tasks in docker environment?

Usecase:

I have a Flow composed of 3 tasks. First two are pretty simple, and can be done without using any external python library. The third one requires some external python libraries and a few system dependencies to be installed. Is there a way to run the first two tasks on the host environment, and the third in docker?

The link here

<https://docs.prefect.io/tutorials/docker/#prerequisites>

explains how to run a flow in a docker environment, but talks nothing about task level docker support

Since you can only deploy a flow, not a task, you can’t run tasks in different environments. However, you can use <https://docs.prefect.io/concepts/flows/#composing-flows|subflow> to do the same thing

Hi <@U03GUVCV8DS> Thanks for the prompt reply. However, this doesn't solves the problem I'm facing.

Here's my question in context of subflows: Can I deploy the main flow in one environment, and the subflow in another? I want to invoke my main flow in host environment, but my subflow should be invoked in `docker` environment.

You could have different subflows in different environments. For your use case, you can run the first two subflows on the host environment, and the third in docker