<@ULVA73B9P> prefect-version: 3.4.4 name: {deploym...
# ask-marvin
r
@Marvin prefect-version: 3.4.4 name: {deployment_name} pull: - prefect.deployments.steps.git_clone: repository: {repo_link} branch: {branch_name} deployments: - name: {deployment_name} entrypoint: {file.py:function_name} work_pool: name: {work_pool_name} job_variables: env: PYTHONPATH: {python_path} this is my yaml file but i want to create deployment using a separate .py file apart from the file.py file how do i do that
m
thought for 8.9 seconds
Let me help you create a deployment programmatically using Python instead of YAML. Here's how you can create the equivalent deployment using a separate Python file:
Copy code
python
from prefect import deploy
from prefect.deployments import DeploymentImage
from prefect.filesystems import GitHub

# Create the deployment
deployment = deploy.Deployment.build_from_flow(
    flow_location="{file.py:function_name}",  # Your flow's location
    name="{deployment_name}",
    work_pool_name="{work_pool_name}",
    path="{python_path}",  # This will set PYTHONPATH
    source=GitHub.from_repository(
        repository="{repo_link}",
        reference="{branch_name}"
    )
)

if __name__ == "__main__":
    deployment.apply()
Save this in a new file (e.g.,
create_deployment.py
) and run it with:
Copy code
bash
python create_deployment.py
A few important notes: 1. Replace all the placeholders in curly braces with your actual values. 2. The
flow_location
should point to your flow function using the format "path/to/file.py:function_name" 3. If you need additional environment variables, you can add them like this:
Copy code
python
deployment = deploy.Deployment.build_from_flow(
    # ... other parameters ...
    job_configuration={
        "env": {
            "PYTHONPATH": "{python_path}",
            "OTHER_VAR": "value"
        }
    }
)
4. If your repository is private, you'll need to provide authentication credentials. You can do this by setting up a GitHub token in your Prefect environment. This approach gives you the same functionality as your YAML file but allows you to create the deployment programmatically. It also gives you more flexibility to add logic or conditions if needed. Let me know if you need any clarification on specific parts of this setup!
r
@Marvin give me code for prefect>=3.4.4
m
thought for 10.9 seconds
Ah, let me provide you with the correct code for Prefect 3.x (≥3.4.4). The previous response used some deprecated methods. Here's the correct way to create a deployment programmatically:
Copy code
python
from prefect import flow
from prefect.deployments import Deployment
from prefect.filesystems import GitHub

# Create a deployment
deployment = Deployment(
    name="{deployment_name}",
    flow_name="{function_name}",  # The name of your flow function
    work_pool_name="{work_pool_name}",
    path="{file.py}",  # Path to your flow file relative to repo root
    job_variables={
        "env": {
            "PYTHONPATH": "{python_path}"
        }
    }
).from_source(
    source="{repo_link}",
    branch="{branch_name}"
)

if __name__ == "__main__":
    deployment.apply()
Save this in a separate file (e.g.,
create_deployment.py
) and run it:
Copy code
bash
python create_deployment.py
Key differences from your YAML: 1. The
entrypoint
is split into
path
(the file) and
flow_name
(the function name) 2. Git clone is handled automatically by
.from_source()
3. Environment variables are set through
job_variables
For example, if your actual values were:
Copy code
python
deployment = Deployment(
    name="my-deployment",
    flow_name="my_flow",
    work_pool_name="my-pool",
    path="flows/main.py",
    job_variables={
        "env": {
            "PYTHONPATH": "/path/to/project"
        }
    }
).from_source(
    source="<https://github.com/username/repo>",
    branch="main"
)
Additional options you might find useful: 1. If you need more environment variables:
Copy code
python
job_variables={
    "env": {
        "PYTHONPATH": "{python_path}",
        "OTHER_VAR": "value",
        "ANOTHER_VAR": "value2"
    }
}
2. If you want to add tags or schedule:
Copy code
python
deployment = Deployment(
    # ... other parameters ...
    tags=["tag1", "tag2"],
    interval="0 0 * * *"  # Cron schedule
)
3. If your repository is private, you'll need to configure Git authentication through Prefect's settings. This approach gives you the same functionality as your YAML file but with the flexibility of Python. You can add conditions, loops, or other logic as needed.
r
@Marvin Deployment class is also depricated
m
thought for 8.5 seconds
Let me help you with the current, non-deprecated way to create deployments in Prefect 3.4.4 and above. You're right - we should use the
deploy
function instead:
Copy code
python
from prefect import deploy

if __name__ == "__main__":
    deployment = deploy(
        name="{deployment_name}",
        flow_path="{file.py}",  # Path to your flow file relative to repo root
        flow_name="{function_name}",  # The name of your flow function
        work_pool_name="{work_pool_name}",
        job_variables={
            "env": {
                "PYTHONPATH": "{python_path}"
            }
        }
    ).from_source(
        source="{repo_link}",
        branch="{branch_name}"
    )
For example, with actual values:
Copy code
python
from prefect import deploy

if __name__ == "__main__":
    deployment = deploy(
        name="my-deployment",
        flow_path="flows/main.py",
        flow_name="my_flow",
        work_pool_name="my-pool",
        job_variables={
            "env": {
                "PYTHONPATH": "/path/to/project"
            }
        }
    ).from_source(
        source="<https://github.com/username/repo>",
        branch="main"
    )
Additional options you might want to use: 1. Adding tags and schedule:
Copy code
python
deployment = deploy(
    # ... other parameters ...
    tags=["tag1", "tag2"],
    interval="0 0 * * *"  # Cron schedule
)
2. Adding more environment variables:
Copy code
python
job_variables={
    "env": {
        "PYTHONPATH": "/path/to/project",
        "OTHER_VAR": "value",
        "ANOTHER_VAR": "value2"
    }
}
Save this in a separate Python file (e.g.,
create_deployment.py
) and run it:
Copy code
bash
python create_deployment.py
This is the current, recommended way to create deployments programmatically in Prefect 3.4.4+. The
deploy
function provides a cleaner, more straightforward API compared to the previous approaches.