https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> Do you have documentation for how to ...
# ask-marvin
d
@Marvin Do you have documentation for how to deploy an automation to Prefect 3.x? I know how to create the automation using the Prefect 3.x GUI, but I can't find any documentation explaining how to automate this process.
m
✅ thought for 76.9 seconds
d
@Marvin Do you have documentation for how to deploy an automation to Prefect 3.x? I know how to create the automation using the Prefect 3.x GUI, but I can't find any documentation explaining how to automate this process.
m
✅ thought for 43.1 seconds
Ah yes, deploying flows programmatically in Prefect 3.x. There are two main approaches: 1. Using Python API (Recommended):
Copy code
python
from prefect import flow

@flow
def my_flow():
    print("Hello from my flow!")

# Deploy the flow
my_flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool",
    # Optional: specify a schedule
    schedule={"interval": "1h"},
    # Optional: add tags
    tags=["prod"]
)
2. Using 
prefect.yaml
Configuration: Create a 
prefect.yaml
file in your project:
Copy code
yaml
deployments:
  - name: my-deployment
    entrypoint: path/to/flow.py:my_flow
    work_pool:
      name: my-work-pool
    schedule:
      interval: 1h
    tags:
      - prod
Then deploy using the CLI:
Copy code
bash
prefect deploy
For remote code (like GitHub), you can use:
Copy code
python
from prefect import flow
from prefect.deployments import deploy

flow.from_source(
    "<https://github.com/your-username/your-repo>",
    entrypoint="path/to/flow.py:my_flow"
).deploy(
    name="my-deployment",
    work_pool_name="my-work-pool"
)
A few important notes: - You'll need a work pool set up before deploying - The deployment name must be unique within your workspace - Make sure your code and dependencies are accessible to the worker that will run the flow For more details, check out: - Deployments Concept Documentation - prefect.yaml Configuration Guide - Work Pools and Workers Guide Would you like me to elaborate on any particular aspect of the deployment process?
d
@Marvin Okay, so that is the documentation for deployments. Where is the documentation for automating automations creation?
m
✅ thought for 43.8 seconds
Ah yes, deploying flows programmatically in Prefect 3.x is quite straightforward. Here's how you can automate the deployment process: 1. Basic Deployment:
Copy code
python
from prefect import flow
from prefect.client.orchestration import get_client

@flow
def my_flow():
    print("Hello from my flow!")

# Deploy the flow
my_flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool"
)
2. Deployment from Source Control:
Copy code
python
from prefect.flows import Flow

# Deploy from a GitHub repository
flow = Flow.from_source(
    source="<https://github.com/username/repo>",
    entrypoint="path/to/file.py:flow_function"
).deploy(
    name="my-deployment",
    work_pool_name="my-work-pool"
)
You can enhance your deployment with additional configurations:
Copy code
python
my_flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool",
    schedule={"interval": "1h"},  # Run every hour
    tags=["prod"],
    parameters={"key": "value"},
    description="My production deployment"
)
Key deployment options include: - `name`: The deployment name (required) - `work_pool_name`: The work pool to run the deployment in - `schedule`: Define when the flow should run - `tags`: Add metadata tags - `parameters`: Set default parameters - `description`: Add a description - `version`: Specify a version - `triggers`: Configure event-based triggers For more detailed information, you can refer to: - Deployment via Python Guide - Deployments Concept Documentation Would you like me to elaborate on any specific aspect of the deployment process?
n
if you edit messages, it sends another slack event
so that's why it sent again
👀 1
m
Yes, you can programmatically create automations in Prefect 3.x using either the Python SDK or the REST API. Here's how: Using the Python SDK:
Copy code
python
from prefect.client import get_client
from prefect.client.schemas.objects import AutomationCore, Trigger, RunDeployment

async def create_automation():
    client = get_client()
    
    # Define the automation
    automation = AutomationCore(
        name="My Automated Trigger",
        description="Triggers when a deployment succeeds",
        enabled=True,
        triggers=[
            Trigger(
                match_related_deployment_id="your-deployment-id",
                match_state_name="COMPLETED"
            )
        ],
        actions=[
            RunDeployment(
                deployment_id="target-deployment-id"
            )
        ]
    )
    
    # Create the automation
    await client.create_automation(automation)
Using the REST API:
Copy code
python
import requests

def create_automation():
    api_url = "<https://api.prefect.cloud/api/v1/automations>"
    headers = {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    }
    
    automation_data = {
        "name": "My Automated Trigger",
        "description": "Triggers when a deployment succeeds",
        "enabled": True,
        "triggers": [{
            "match_related_deployment_id": "your-deployment-id",
            "match_state_name": "COMPLETED"
        }],
        "actions": [{
            "type": "run-deployment",
            "deployment_id": "target-deployment-id"
        }]
    }
    
    response = <http://requests.post|requests.post>(api_url, json=automation_data, headers=headers)
    return response.json()
You can find more details in: - Creating Automations Guide - Automations REST API Reference The key components you'll need to specify are: 1. A name for the automation 2. The trigger conditions (what events will trigger the automation) 3. The actions to take when triggered 4. Whether the automation should be enabled immediately Would you like me to provide more specific examples for certain types of triggers or actions?
n
you're probably looking for the 
Automation
object or 
triggers
kwarg in 
.serve
or 
.deploy
d
@Nate Yeah, I was trying to figure out if I can add automations somewhere in my prefect.yaml. I am using the Prefect CLI to update our Prefect deployments, and I'm trying to figure out how to do something similar with automations.
I think the triggers kwarg is what I need to update my deployments with, maybe?
n
i think mintlify is having issues rn with header links but its on that page
d
@Nate Amazing! Thank you, I was struggling to find that page. I'll do some research and see if I can update our CI/CD process.
At least, I think that's what I need? I'm trying to create an automation to cancel a flow run if the flow run is in the running state for longer than 5 minutes.
@Marvin If I want to create an automation for canceling a flow run that has been running for longer than 5 minutes, is there a YAML configuration that I can update, or do I have to use the Prefect CLI and GUI to create these automations?
m
✅ thought for 43.7 seconds
Based on my research, there isn't currently a YAML-based configuration method for creating automations in Prefect 3.x. Automations need to be created through one of these methods: 1. The Prefect UI 2. The Python SDK 3. The REST API For your specific use case of canceling long-running flows, here's how you could create it programmatically:
Copy code
python
from prefect.client import get_client
from prefect.client.schemas.objects import AutomationCore, Trigger, CancelFlowRun

async def create_timeout_automation():
    client = get_client()
    
    automation = AutomationCore(
        name="Cancel Long-Running Flows",
        description="Cancels flows running longer than 5 minutes",
        enabled=True,
        triggers=[
            Trigger(
                match_state_name="RUNNING",
                match_state_duration_s=300  # 5 minutes in seconds
            )
        ],
        actions=[
            CancelFlowRun()
        ]
    )
    
    await client.create_automation(automation)
Or using the REST API:
Copy code
python
import requests

def create_timeout_automation():
    api_url = "<https://api.prefect.cloud/api/v1/automations>"
    headers = {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    }
    
    automation_data = {
        "name": "Cancel Long-Running Flows",
        "description": "Cancels flows running longer than 5 minutes",
        "enabled": True,
        "triggers": [{
            "match_state_name": "RUNNING",
            "match_state_duration_s": 300
        }],
        "actions": [{
            "type": "cancel-flow-run"
        }]
    }
    
    response = <http://requests.post|requests.post>(api_url, json=automation_data, headers=headers)
    return response.json()
While it would be convenient to have YAML configuration support for automations (similar to how deployments work with 
prefect.yaml
), this functionality isn't currently available. If you need to create multiple automations or want to version control them, you might want to create a Python script that uses one of the above methods to set up your automations programmatically. Would you like me to provide more details about specific trigger conditions or action types you can use with these methods?
n
the format doesn't matter outside of convenience. you can use yaml in prefect.yaml, dict in python SDK or JSON against the rest api, they all support all the same things
d
@Nate For this specific automation trigger of cancelling a flow run, would I still be able to use deployment triggers? The documentation states that:
Deployment triggers are a special case of automations where the configured action is always running a deployment.
But, I want my configuration action to be canceling a flow run?
n
oh oh i see. sorry, was too hasty bc you don't want the 
Run Deployment
action, and you want a general automation, there is indeed a gap for yaml here i think we should add this in the automations cli, like
Copy code
prefect automation create foo.yaml # foo.json
so for now yeah marvin is right that you'd want the SDK or the REST api directly
d
@Nate Okay, thanks for confirming. The Prefect CLI doesn't have a create option, but is this something on the Prefect 3.x roadmap? https://docs.prefect.io/v3/api-ref/cli/automation
n
correct i am saying that's something we should add
d
@Nate Got it, thank you! I appreciate the help 🙏 Should I open an issue on the Prefect GitHub for tracking? Or is this already being tracked elsewhere?
n
if you are willing, an issue would be helpful for tracking!
d
@Nate Done! Thanks again for confirming the functionality 🙏 https://github.com/PrefectHQ/prefect/issues/18451
n
will get cleaned up and reviewed but feel free to give feedback while the iron is hot
d
added a comment! thanks for addressing the issue so quickly 🙏 https://github.com/PrefectHQ/prefect/pull/18452#issuecomment-3046366225
n
oh yea great call, arguably the file or json trigger ref should be positional and you should pass many but each file should def be able to define many
will update
🙌 1
🙏 1
d
thanks again for all these updates! looks good to me. made a small follow-up comment about the docs. https://github.com/PrefectHQ/prefect/pull/18452#issuecomment-3046432068
@Nate just checked out the update, looks awesome 🙌 and thanks for jumping on it so quickly! is there anywhere I can drop positive feedback or kudos?
n
sweet! there's a #C048ZJCV8F4 if you're so inclined but no need, happy to help catjam
🙏 1
🙌 1
catjam 1
2 Views
Open in Slack
PreviousNext
Powered by