https://prefect.io logo
#prefect-community
Title
# prefect-community
x

Xavier Babu

09/26/2022, 2:36 PM
When I use REST API with Scheduler enabled (e.g., RRULE), I get the following error in Prefect Orion 2.4. It was working fine in 2.0. Even though I have dedicated storage directory and REST API way of running workflow is pointing to default /tmp folder, not the one I have it using the parameter PREFECT_LOCAL_STORAGE_PATH. But If I run using .yml, it uses the path specified in PREFECT_LOCAL_STORAGE_PATH and it works fine. Please let me know what am I missing in the payload? Flow could not be retrieved from deployment. Traceback (most recent call last): File "<frozen importlib._bootstrap_external>", line 879, in exec_module File "<frozen importlib._bootstrap_external>", line 1016, in get_code File "<frozen importlib._bootstrap_external>", line 1073, in get_data FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpz0cmn1ddprefect/AOPS_SQL_Workflow_v1.py'
k

Khuyen Tran

09/26/2022, 2:42 PM
Hi @Xavier Babu, thanks for raising the issue. My teammates might be able to help. Can you move the error to the comment so that it is easier for others to follow other threads?
x

Xavier Babu

09/26/2022, 2:45 PM
Payload used: 2022-09-24 164054.170 DEBUG 18568 --- [http-nio-5080-exec-962] com.aops.waves.service.FlowService : jsonObject.toJSONString(): { "parameter_openapi_schema": { "type": "object", "title": "Parameters", "properties": { "kwargs": "{\"type\": \"string\", \"title\":\"kwargs\"}" }, "required": [ "kwargs" ] }, "infrastructure_document_id": "736c6e6f-6c03-40fa-8ea4-71f946a05343", "infra_overrides": {}, "description": "AOPS_SQL_Workflow_DS_12", "version": "1", "work_queue_name": "waves_q", "tags": [ "waves_q" ], "path": "/localpart0/aop-shared/WAVES/workflows/", "schedule": { "rrule": "DTSTART20220924T124300\nRRULEFREQ=HOURLY;INTERVAL=1;COUNT=1;UNTIL=20220924T124500", "timezone": "US/Eastern" }, "flow_id": "d7202f6b-b929-4139-9e4d-73e2636a3fe0", "entrypoint": "AOPS_SQL_Workflow_v1.py:AOPS_SQL_Workflow", "name": "AOPS_SQL_Workflow_DS_12", "parameters": { "kwargs": { "sqltype": 1, "date_range": "CURRENT_DATE - INTERVAL '1 months'", "dbname": "gpprod", "selection": "count(*)", "flow_name": "AOPS_SQL_Workflow", "rpt_flag": "1", "tab1": "whse.dim_company", "schd_run_name": "AOPS_SQL_Workflow_DS_12", "sql": "SELECT {selection} FROM {tab1} WHERE show_in_report_flag = {rpt_flag} and (creation_date > ({date_range}));" } }, API Response: { "id": "ba109826-6846-4e0e-815b-ee905c593dab", "created": "2022-09-23T194943.563714+00:00", "updated": "2022-09-24T164054.208488+00:00", "name": "AOPS_SQL_Workflow_DS_12", "version": "1", "description": "AOPS_SQL_Workflow_DS_12", "flow_id": "d7202f6b-b929-4139-9e4d-73e2636a3fe0", "schedule": { "rrule": "DTSTART20220924T124300\nRRULEFREQ=HOURLY;INTERVAL=1;COUNT=1;UNTIL=20220924T124500", "timezone": "US/Eastern" }, "is_schedule_active": true, "infra_overrides": {}, "parameters": { "kwargs": { "sql": "SELECT {selection} FROM {tab1} WHERE show_in_report_flag = {rpt_flag} and (creation_date > ({date_range}));", "tab1": "whse.dim_company", "dbname": "gpprod", "sqltype": 1, "rpt_flag": "1", "flow_name": "AOPS_SQL_Workflow", "selection": "count(*)", "date_range": "CURRENT_DATE - INTERVAL '1 months'", "schd_run_name": "AOPS_SQL_Workflow_DS_12" } }, "tags": [ "waves_q" ], "work_queue_name": "waves_q", "parameter_openapi_schema": { "type": "object", "title": "Parameters", "required": [ "kwargs" ], "properties": { "kwargs": "{\"type\": \"string\", \"title\":\"kwargs\"}" } }, "path": "/localpart0/aop-shared/WAVES/workflows/", "entrypoint": "AOPS_SQL_Workflow_v1.py:AOPS_SQL_Workflow", "manifest_path": null, "storage_document_id": null, "infrastructure_document_id": "736c6e6f-6c03-40fa-8ea4-71f946a05343" } "is_schedule_active": "true" }
Moved.
🙏 2
If we run a workflow for every minute for 5 minutes, it fails sporadically by showing the following error: Flow could not be retrieved from deployment. Traceback (most recent call last): File "<frozen importlib._bootstrap_external>", line 879, in exec_module File "<frozen importlib._bootstrap_external>", line 1016, in get_code File "<frozen importlib._bootstrap_external>", line 1073, in get_data FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp7v3me6ytprefect/AOPS_SQL_Workflow_v1.py' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/engine.py", line 257, in retrieve_flow_then_begin_flow_run flow = await load_flow_from_flow_run(flow_run, client=client) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/client/orion.py", line 82, in with_injected_client return await fn(*args, **kwargs) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/deployments.py", line 70, in load_flow_from_flow_run flow = await run_sync_in_worker_thread(import_object, str(import_path)) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 57, in run_sync_in_worker_thread return await anyio.to_thread.run_sync(call, cancellable=True) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 193, in import_object module = load_script_as_module(script_path) File "/localpart0/aop-shared/WAVES/prefect2/lib/python3.10/site-packages/prefect/utilities/importtools.py", line 156, in load_script_as_module raise ScriptError(user_exc=exc, path=path) from exc prefect.exceptions.ScriptError: Script at 'AOPS_SQL_Workflow_v1.py' encountered an exception
Khuyen, Any quick tip to resolve this issue? It has stopped the entire DEV work. Impacting our application release date. Please help.
k

Khuyen Tran

09/26/2022, 6:15 PM
Hmm. I’m not sure how to solve this issue. Do you mind opening this issue on Prefect GitHub so the engineers can take a look at this?
x

Xavier Babu

09/27/2022, 2:24 PM
Good morning Khuyen. I opened a issue/defect yesterday. Since it made our entire system down, could you please expedite if possible? https://github.com/PrefectHQ/prefect/issues/6979
3 Views