Hi all, what is the recommended way to deploy flows that come pre-built in 3rd party packages? Think e.g.

scikit-learn

providing a flow to train a machine learning model and you can deploy it with your own model class like

Deployment.build_from_flow(flow=sklearn.flows.training, …, parameters={"model": SomePythonClass})

Is that even feasible? I guess another way to look at it is “prefect collection but with flows, not tasks”

Collections can have flows in them too like trigger_dbt_cloud_job_run_and_wait_for_completion (or any other integration, like DaskTaskRunner).

yeah I saw that but my understanding is that they act like tasks anyway, that is you still need to create your own main flow that encapsulates them. Or do you have an example where they get deployed directly?

you can trigger them directly https://github.com/PrefectHQ/prefect-dbt/blob/main/prefect_dbt/cloud/jobs.py#L212-L222

yes but I’m talking about deployment, not just one-off runs

sorry I don’t quite understand. although it’s in a collection, it’s still a flow, and that means you can deploy it like any other flow, once you import it

when I do this

from prefect.deployments import Deployment
from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import trigger_dbt_cloud_job_run_and_wait_for_completion

Deployment.build_from_flow(
    flow=trigger_dbt_cloud_job_run_and_wait_for_completion,
    name="test-deployment"
)

I get this error

ValueError: '/Users/michal.augoff/anaconda3/envs/prefect2-demo/lib/python3.8/site-packages/prefect_dbt/cloud/jobs.py' does not start with '/Users/michal.augoff/PycharmProjects/prefect-demo/test-deployment'

test.py:

from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import trigger_dbt_cloud_job_run_and_wait_for_completion

and then cli:

prefect deployment build test.py:trigger_dbt_cloud_job_run_and_wait_for_completion --name my_deployment

does this work for you?

yes it does. So I guess the python equivalent would be explicitly specifying

entrypoint=f"{Path(__file__).name}:trigger_dbt_cloud_job_run_and_wait_for_completion"

for

Deployment.build_from_flow