@Marvin I want to deploy a flow and I do a

prefect deployment run "knowbe4_extract/extract-knowbe4-dlt"

, it tells me

Deployment 'knowbe4_extract/extract-knowbe4-dlt' not found!

. I don’t understand, I thought it would create such a deployment, How can I fix this?

The command

prefect deployment run "knowbe4_extract/extract-knowbe4-dlt"

expects the specified deployment to already exist, it does not create a new deployment. To fix this issue, you need to create a deployment first before you can run it. To create a deployment, you need to: 1. Configure a

deployment.yaml

file for your flow. 2. Use the Prefect CLI command

prefect deployment apply <path-to-your-deployment-yaml>

to create the deployment on the API. For example:

$ prefect deployment apply ./catfacts_flow-deployment.yaml
Successfully loaded 'catfact'
Deployment '76a9f1ac-4d8c-4a92-8869-615bec502685' successfully created.

Once your deployment is created successfully, you can then use the command

prefect deployment run "your-deployment-name"

to run the flow associated with that deployment. Refer to the Deployments documentation for detailed steps on how to create and apply a deployment. Additionally, you can find out more about how to define, apply, and run deployments in the Prefect deployment guide.

@Marvin But I managed to create a deployment for another flow without a

deployment.yaml

. How is it possible?

It is possible to create a deployment for a flow directly from a Python script without needing a

deployment.yaml

file, using the Prefect API. This can be done through the

prefect.deployments.Deployment

class. Here's how you can create a deployment from a Python object:

python
from your_project.flows import your_flow
from prefect.deployments import Deployment

deployment = Deployment.build_from_flow(
    flow=your_flow,
    name="example-deployment",
    version=1,
    work_queue_name="demo",
    work_pool_name="default-agent-pool",
)

deployment.apply()

This snippet creates a new deployment using any necessary configuration defaults for your imported flow. You can customize the deployment with parameters such as

name

,

version

,

work_queue_name

, and

work_pool_name

according to your setup and preferences. Furthermore, if you have a pre-defined storage block and need to apply infrastructure overrides, you can do so within the Deployment class as well:

python
from your_project.flows import your_flow
from prefect.deployments import Deployment
from prefect.filesystems import S3

# Load a pre-defined block
storage = S3.load("dev-bucket")

deployment = Deployment.build_from_flow(
    flow=your_flow,
    name="s3-example",
    version=2,
    work_queue_name="custom-work-queue",
    storage=storage,
    # Other configurations...
)

deployment.apply()

For more information on creating deployments from Python objects, you can refer to the documentation on creating a deployment from a Python object.

@Marvin, this doesn’t make sense. You refer to “Block Based Deployments” which is not the recommended way anymore. I want the “Work Pool” based deployment way of creating deployments.

In Prefect 2.0, deploying flows with work pools is different from the approach of using block-based deployments. Work pools are used to manage flow runs, and you can create a deployment without a

deployment.yaml

file using the Prefect API and CLI. Here are the general steps to create a work pool based deployment: 1. Set up a work pool: Use Prefect Cloud or Prefect Core to create a work pool that your deployment will use. For example, to create a managed work pool with Prefect Cloud, you would run:

prefect work-pool create my-managed-pool --type prefect:managed

2. Deploy your flow: Once you have a work pool, you can use Prefect's deployment capability to deploy your flow. Specify the work pool you wish to use when creating your deployment. For detailed steps on setting up a work pool, refer to the Work Pools tutorial. For more on work pools and job variables customization, you can visit the Work Pools concepts page. It is also noted that if you specify overrides in the

work_pool.job_variables

section of a

deployment.yaml

, this could customize your deployment settings. However, since you're asking about creating deployments without a

deployment.yaml

, you may want to leverage the Prefect 2.x features that allow more dynamic creation of deployments where everything can be specified at runtime. For further details on the migration to Prefect 2 and the changes in how deployments are handled compared to Prefect 1, you can reference the Migration Guide. If you've managed to create a deployment for another flow without a

deployment.yaml

file, you have likely used Prefect 2.x's capabilities of dynamic deployment directly through the CLI or a Python script that does not require the static yaml-based configuration.

marvin's off the rails here i would just

prefect deployment ls

I figured it out in the end 🤣 I just had to run my python file itself as it has a

xyz.deploy(...)

in it