Shayne Hodge

01/12/2023, 8:57 PM
Hello, We're trying to migrate from Prefect 1 to 2, but we want to run both concurrently on the same data to verify the output is the same. Both lib versions can't be installed at once, so I'm using the REST API to trigger jobs. We have a flow set up that works and that we've tested launching through the UI. I've followed this guide, and my code looks something like this:
data = {
        'parameters': params, 
        'flow_id': 'cfd9e6db...',
        'deployment_id': '24583f30...4',
        'infrastructure_document_id': '0181dc52...',
        # tried without the next key entirely, and in "PENDING"
        "state": {"type": "RUNNING"}
    r = <|>(PREFECT2_URL, json=data, headers={'Authorization': f'Bearer {api_key}'})
The flow is created, but it doesn't pick up the work queue properly, and so it just stays in the state but nothing happens. See below, the green run was triggered in the UI, the other two were sent in as "SCHEDULED" and "RUNNING", respectively. Any suggestions on getting this to run would be much appreciated. Thanks!
Here's an example response:
    "id": "585e9cad-...",
    "created": "2023-01-12T21:03:29.623120+00:00",
    "updated": "2023-01-12T21:03:29.625422+00:00",
    "name": "modest-ladybug",
    "flow_id": "cfd9e6db-...",
    "state_id": "1e0299c4-...",
    "deployment_id": "24583f30-...",
    "work_queue_name": None,
    "flow_version": None,
    "parameters": {"documentset_id": "08eff4a9-..."},
    "idempotency_key": None,
    "context": {},
    "empirical_policy": {"max_retries": 0, "retry_delay_seconds": 0.0, "retries": None,
                         "retry_delay": None, "pause_keys": [], "resuming": False},
    "tags": [],
    "parent_task_run_id": None,
    "state_type": "RUNNING",
    "state_name": "Running",
    "run_count": 1,
    "expected_start_time": "2023-01-12T21:03:29.622674+00:00",
    "next_scheduled_start_time": None,
    "start_time": "2023-01-12T21:03:29.622674+00:00",
    "end_time": None,
    "total_run_time": 0.0,
    "estimated_run_time": 0.053743,
    "estimated_start_time_delta": 0.0,
    "auto_scheduled": False,
    "infrastructure_document_id": "0181dc52-...",
    "infrastructure_pid": None,
    "created_by": {"id": "48387560-...", "type": "USER", "display_value": "shaynetomecom"},
    "work_pool_name": None,
    "work_pool_queue_name": None,
    "state": {"id": "1e0299c4-545e-4939-b75e-7b1a88fdf182", "type": "RUNNING", "name": "Running", "timestamp": "2023-01-12T21:03:29.622674+00:00",
            "message": None,    "data": None, "state_details": {
                    "flow_run_id": "585e9cad-...", "task_run_id": None, "child_flow_run_id": None,
                    "scheduled_time": None, "cache_key": None, "cache_expiration": None, "untrackable_result": False, "pause_timeout": None,
                    "pause_reschedule": False, "pause_key": None}

Mason Menges

01/13/2023, 2:24 PM
Hey @Shayne Hodge Do you have the agent logs for these flow runs? are they just running locally through the deployment, generally speaking if the flows are getting stuck it indicates and issue in the execution environment

Shayne Hodge

01/13/2023, 5:31 PM
So using prefect cloud, with agents in ECS. Agent logs show absolutely nothing, it doesn't appear anything is reaching them. If I kick it off from the UI logs appear.
The consistent difference between runs from the REST API vs from the UI is the API ones have no work queue attached

Nimesh Kumar

01/24/2023, 5:40 PM
Hi, shayne I am also trying to trigger my flow from outside using api post request, can you share this code and the PREFECT2_URL you have used here. Thanks