https://prefect.io logo
Title
r

Ross Teach

09/15/2022, 10:56 AM
Hello all. I recently updated from Prefect 2.1.0 to 2.3.2. After updating, I re-deployed my existing deployments. I am now seeing flows being scheduluded twice. I do not see duplicate deployments listed in the UI or API. Based on the created dates, it appears Prefect is still scheduling using the 2.1.0 deployments as well as the 2.3.2 deployments. The 'old' flow runs have null value the deployment_id field. FYI I am using Prefect cloud. Has anyone seen anything like this and/or have a recommended way of purging what seems to be previous Prefect deployments still being scheduled?
1
{
  "id": "ffc2832b-959b-4950-b08f-4a710b850c35",
  "created": "2022-09-06T13:11:33.108901+00:00",
  "updated": "2022-09-06T13:11:28.970538+00:00",
  "name": "enormous-toucan",
  "flow_id": "d3f7aed7-f1a7-4d10-9903-04b7f20bb846",
  "state_id": "2b539407-f42e-4a5d-b047-568b293b6dbc",
  "deployment_id": null,
  "work_queue_name": "prod",
  "flow_version": null,
  "parameters": {},
  "idempotency_key": "scheduled 2f8d28e6-9b39-497f-a1cf-9d656b5cd768 2022-10-16T06:00:00-04:00",
  "context": {},
  "empirical_policy": {
    "max_retries": 0,
    "retry_delay_seconds": 0,
    "retries": null,
    "retry_delay": null
  },
  "tags": [
    "auto-scheduled",
    "prod",
    "airbyte",
    "google"
  ],
  "parent_task_run_id": null,
  "state_type": "SCHEDULED",
  "state_name": "Scheduled",
  "run_count": 0,
  "expected_start_time": "2022-10-16T10:00:00+00:00",
  "next_scheduled_start_time": "2022-10-16T10:00:00+00:00",
  "start_time": null,
  "end_time": null,
  "total_run_time": 0,
  "estimated_run_time": 0,
  "estimated_start_time_delta": 0,
  "auto_scheduled": true,
  "infrastructure_document_id": "56c4c095-188f-4a89-9a6e-ccc18eda986b",
  "state": {
    "id": "2b539407-f42e-4a5d-b047-568b293b6dbc",
    "type": "SCHEDULED",
    "name": "Scheduled",
    "timestamp": "2022-09-06T13:11:30.485515+00:00",
    "message": "Flow run scheduled",
    "data": null,
    "state_details": {
      "flow_run_id": null,
      "task_run_id": null,
      "child_flow_run_id": null,
      "scheduled_time": "2022-10-16T06:00:00-04:00",
      "cache_key": null,
      "cache_expiration": null,
      "untrackable_result": false
    }
  }
}
a

Anna Geller

09/15/2022, 11:00 AM
I'd recommend either: 1. Toggle schedule off and back on 2. Create a deployment with a new name 3. Better: remove and recreate a deployment the latest release has an update mechanism so that if you already have a deployment, Prefect will only update those arguments that changed rather than doing it in a way that deletes deployment and creates a new one from scratch from the latest config
r

Ross Teach

09/15/2022, 11:02 AM
Thank you, so likely deleting all my existing deployments and then re-creating them should do the job?
a

Anna Geller

09/15/2022, 11:24 AM
absolutely! here is a script that will make it easy:
import asyncio
from prefect.client import get_client


async def remove_all_deployments():
    client = get_client()
    deployments = await client.read_deployments()
    for deployment in deployments:
        print(f"Deleting deployment: {deployment.name}")
        await client.delete_deployment(deployment.id)
        print(f"Deployment with UUID {deployment.id} deleted")


if __name__ == "__main__":
    asyncio.run(remove_all_deployments())
once you run it, you have a clean slate and all deployments are gone and you can start from scratch
r

Ross Teach

09/15/2022, 11:27 AM
Thank you, very helpful. I will incorporate this into my CI
👍 1