https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> How do I create, build and run prefec...
# ask-marvin
m
@Marvin How do I create, build and run prefect deployments dynamically using python?
m
✅ thought for 112.8 seconds
Based on the research, I'll provide you with a comprehensive guide on how to create and run deployments programmatically in Prefect 3.x. Here's how to do it: 1. Basic Deployment Creation
Copy code
python
from prefect import flow
from prefect.deployments import Deployment

@flow
def my_flow(name: str = "world"):
    print(f"Hello {name}!")

# Create deployment from a local flow
deployment = my_flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool",
    tags=["prod"],
)
2. Deployment from Source (e.g., GitHub)
Copy code
python
from prefect.deployments import Deployment
from prefect.filesystems import GitHub

# Deploy from a GitHub repository
deployment = Deployment.from_source(
    source="<https://github.com/owner/repo>",
    entrypoint="path/to/file.py:flow_function",
    name="my-deployment",
    work_pool_name="my-work-pool"
)
3. Running a Deployment
Copy code
python
from prefect.deployments import run_deployment

# Run the deployment asynchronously
deployment_run = await run_deployment(
    name="my-flow/my-deployment",
    parameters={"name": "Prefect"}
)
4. Creating with Custom Settings
Copy code
python
from prefect import flow

@flow
def my_flow():
    print("Hello from my flow!")

# Deploy with additional configurations
deployment = my_flow.deploy(
    name="my-deployment",
    work_pool_name="my-work-pool",
    tags=["prod"],
    parameters={"default_param": "value"},
    description="My deployment description",
    version="1.0.0",
    schedule={"interval": "1h"},  # Run every hour
)
Key Points to Remember: 1. You need to have a work pool created before deploying. You can create one via the UI or programmatically. 2. The 
work_pool_name
parameter is required when creating a deployment. 3. You can set schedules, parameters, and other configurations during deployment creation. 4. When deploying from source (like GitHub), make sure to specify the correct entrypoint path to your flow. To run your deployment, you'll need to: 1. Ensure you have a worker running for the specified work pool 2. Use 
run_deployment()
to trigger the deployment programmatically, or 3. Let the scheduler run it according to any defined schedule Here's how to set up a work pool programmatically if needed:
Copy code
python
from prefect.workpools import WorkPool

work_pool = WorkPool(
    name="my-work-pool",
    work_queue_name="default",
    type="process"  # or "kubernetes", "docker", etc.
)
work_pool.save()
Important Notes: - Always ensure your work pool exists before creating deployments - The deployment name must be unique within the context of your flow - When running deployments, use the format 
"flow_name/deployment_name"
as the identifier - Remember that any dependencies your flow needs should be available in the environment where it will run Would you like me to elaborate on any particular aspect of deployment creation or management?
m
Why do I get this error when I run my flow: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/websockets/asyncio/client.py", line 541, in __await_impl__ self.connection = await self.create_connection() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/websockets/asyncio/client.py", line 467, in create_connection _, connection = await loop.create_connection(factory, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1046, in create_connection infos = await self._ensure_resolved( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1420, in _ensure_resolved return await loop.getaddrinfo(host, port, family=family, type=type, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 868, in getaddrinfo return await self.run_in_executor( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/websockets/asyncio/client.py", line 539, in __await_impl__ async with asyncio_timeout(self.open_timeout): File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 115, in aexit raise TimeoutError from exc_val TimeoutError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 44, in wrapper return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/prefect/cli/_types.py", line 156, in sync_fn return asyncio.run(async_fn(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/prefect/cli/flow_run.py", line 412, in execute
3 Views
Open in Slack
PreviousNext
Powered by