@Marvin When calling "prefect deployment run" via CLI, can I pass a json file for the parameters?

Currently, there is no direct support mentioned in the Prefect documentation for passing parameters via a JSON file when using the

prefect deployment run

command through the CLI. Instead, parameters for deployments are typically set within the flow script or specified directly via CLI when running a flow or during deployment creation. In some cases, you may create a deployment which generates a YAML file. You can then modify the default values for parameters in that YAML file. If you need to override parameters for custom deployments, altering them directly on the flow function or modifying the deployment YAML file may be the way to go. For ad-hoc flow runs with different parameters, you can provide custom values through the deployment's details page under the Parameters section on the Prefect UI. If you need to pass a complex set of parameters or you have them already in a JSON file, a possible workaround would be to read the JSON file in your flow script and pass the contents as parameters programmatically, or to parse them into the appropriate command-line format for the

prefect deployment run

command.

What is the python equivalent of CLI prefect deployment run?

from prefect.deployments import run_deployment

To run a deployed flow from Python code in Prefect 2.x, you would not directly invoke the deployment as you might with the CLI. Instead, you can import the flow function that you have deployed and execute it just as you would any other function in Python. Here's an outline of the steps you would follow: 1. First, make sure the flow has been deployed. The deployment process registers the flow with Prefect Cloud or Prefect Server and makes it possible to run it. 2. In your Python script or environment where you want to run the flow, import the flow function. 3. Run the flow function with the desired parameters, just like calling any other Python function. Here's a simple example:

python
from your_flow_file import your_flow_function

# Run the flow with any required parameters
your_flow_function(param1=value1, param2=value2)

Do note that running the flow in this way will execute it immediately in your current Python environment. It will not utilize the Prefect work queue or use agents to run the flow, unlike when triggering it via the Prefect Cloud/UI or CLI deployment commands. If you require the flow to be run as a deployment via the Prefect API, rather than directly invoking the flow function in Python, you would need to interact with the Prefect API using a client library or HTTP requests. This would involve authenticating with Prefect Cloud or Prefect Server, creating a flow run in the desired deployment, and then monitoring its progress. This approach typically requires a more advanced setup and understanding of Prefect's API. The Prefect documentation does not provide a detailed guide for running a deployment programmatically through Python outside of invoking the flow function directly, so for more complex scenarios involving deployments, using the provided Prefect CLI or UI tools would be recommended.

Thanks @Nate but actually I'm more interested by the second part of Marvin's answer. I want to launch a deployed flow, on a remote machine, with a somewhat complex set of parameters. Ideally I would do "prefect deployment run ... --params params.json" but that's not possible. So I'm looking for the equivalent in python.

To create a flow run from an existing deployment using HTTP requests in Python, you can use an HTTP client like

httpx

. The following example demonstrates how to trigger a flow with parameters:

python
# First, install the httpx library if you haven't already:
# pip install httpx

import os
import httpx

# Set up the API URL and the authorization header
api_url = os.getenv('PREFECT_API_URL')
headers = {
    "Authorization": f"Bearer {os.getenv('PREFECT_API_KEY')}"
}

# Specify the parameters for your flow run
parameters = {
    "param1": "value1",
    "param2": "value2"
}

# Provide a name for the flow run and the parameters in the payload
payload = {
    "name": "my-new-flow-run-name",  # This is optional
    "parameters": parameters
}

deployment_id = "<my-deployment-uuid>"

# Make an asynchronous POST request to the Prefect API to create a new flow run
async with httpx.AsyncClient() as client:
    response = await <http://client.post|client.post>(
        f"{api_url}/deployments/{deployment_id}/create_flow_run",
        headers=headers,
        json=payload
    )
    response.raise_for_status()

# Check the response for the details of the created flow run
print(response.json())

Make sure to replace

<my-deployment-uuid>

with the actual UUID of the deployment that you want to trigger. In addition, set the parameters as needed for your specific flow. This script assumes that you’ve set the environment variables

PREFECT_API_URL

, which is the URL to the Prefect API, and

PREFECT_API_KEY

, which is your API key for authentication. You'll have to export these variables in your environment or set them in your script before running it. Before running this script, you need to install the

httpx

library if you haven't already. You can install it using pip:

sh
pip install httpx

Finally, note that the example provided is using an asynchronous client (

AsyncClient

), which means you'll need to run this within an asynchronous event loop, such as

asyncio.run

if using it as a standalone script. If you're not familiar with async programming in Python, you might want to use a synchronous client or adapt the code accordingly.