MichaelT
02/17/2025, 11:56 PMmy_deployment = await my_flow.from_source(
source=os.getcwd(),
entrypoint="workflows/status_checker/my_flow.py:my_flow"
)
deployment = await my_deployment.to_deployment(
name=f"my_flow",
schedule={"interval": timedelta(minutes=schedule_interval)}
)
try:
result = await deploy(
deployment,
work_pool_name=work_pool
)
<http://logger.info|logger.info>(f"Deployment result: {result}")
it works but i end up with pull steps like this;
'pull_steps': [{'prefect.deployments.steps.set_working_directory': {'directory': '/tmp/tmps_9qxoh3prefect/my-flows-main'}}],
this will not work as that tmp directory will not exist on the scheduled flow runs on a process worker.
i want it to have pull steps like this;
'pull_steps': [
{
'prefect.deployments.steps.git_clone': {
'branch': 'main',
'repository': '<https://gitlab.com/xxx/my-flows.git>',
'access_token': '{{ prefect.blocks.secret.gitlab-secret-oauth }}'
}
}
],
how can i define the pull_steps when programmatically deploying?
i am using prefect v2 locally hosted.Marvin
02/17/2025, 11:56 PMpull
action is crucial as it defines how Prefect will pull your deployment in remote execution environments. Here's how you can define a custom pull step:
1. Define Pull Steps in `prefect.yaml`: You can specify a prefect.yaml
file to control your deployment configuration. This file will include the pull steps you desire.
2. Programmatic Deployment with Custom Pull Steps:
You can create a deployment programmatically by specifying the pull steps. Here's a conceptual outline:
python
from prefect.deployments import Deployment
my_deployment = await my_flow.from_source(
source=os.getcwd(),
entrypoint="workflows/status_checker/my_flow.py:my_flow"
)
deployment = await my_deployment.to_deployment(
name="my_flow",
schedule={"interval": timedelta(minutes=schedule_interval)},
pull_steps=[
{
'prefect.deployments.steps.git_clone': {
'branch': 'main',
'repository': '<https://gitlab.com/xxx/my-flows.git>',
'access_token': '{{ prefect.blocks.secret.gitlab-secret-oauth }}'
}
}
]
)
result = await deploy(
deployment,
work_pool_name=work_pool
)
<http://logger.info|logger.info>(f"Deployment result: {result}")
3. Reference Documentation:
For more details on deployment configurations and pull steps, you can refer to the Prefect Deployment Guide.
By specifying your pull steps directly when creating the deployment, you ensure that the correct repository and branch are used, with the necessary credentials. This approach should align with your requirement to use a Git repository for deployment rather than a temporary directory.