Hi all, what is the recommended way to deploy flow...
# ask-community
m
Hi all, what is the recommended way to deploy flows that come pre-built in 3rd party packages? Think e.g.
scikit-learn
providing a flow to train a machine learning model and you can deploy it with your own model class like
Copy code
Deployment.build_from_flow(flow=sklearn.flows.training, …, parameters={"model": SomePythonClass})
Is that even feasible? I guess another way to look at it is “prefect collection but with flows, not tasks”
1
a
Collections can have flows in them too like trigger_dbt_cloud_job_run_and_wait_for_completion (or any other integration, like DaskTaskRunner).
m
yeah I saw that but my understanding is that they act like tasks anyway, that is you still need to create your own main flow that encapsulates them. Or do you have an example where they get deployed directly?
a
m
yes but I’m talking about deployment, not just one-off runs
a
sorry I don’t quite understand. although it’s in a collection, it’s still a flow, and that means you can deploy it like any other flow, once you import it
m
when I do this
Copy code
from prefect.deployments import Deployment
from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import trigger_dbt_cloud_job_run_and_wait_for_completion

Deployment.build_from_flow(
    flow=trigger_dbt_cloud_job_run_and_wait_for_completion,
    name="test-deployment"
)
I get this error
Copy code
ValueError: '/Users/michal.augoff/anaconda3/envs/prefect2-demo/lib/python3.8/site-packages/prefect_dbt/cloud/jobs.py' does not start with '/Users/michal.augoff/PycharmProjects/prefect-demo/test-deployment'
👀 1
a
test.py:
Copy code
from <http://prefect_dbt.cloud.jobs|prefect_dbt.cloud.jobs> import trigger_dbt_cloud_job_run_and_wait_for_completion
and then cli:
Copy code
prefect deployment build test.py:trigger_dbt_cloud_job_run_and_wait_for_completion --name my_deployment
does this work for you?
m
yes it does. So I guess the python equivalent would be explicitly specifying
Copy code
entrypoint=f"{Path(__file__).name}:trigger_dbt_cloud_job_run_and_wait_for_completion"
for
Deployment.build_from_flow
👀 1
104 Views