https://prefect.io logo
v

Venkat Ramakrishnan

08/28/2022, 4:04 AM
Hi, I am getting the following error when I run Prefect agent looking for flows deployed from a python script running on a Windows laptop. Further details in the thread. 092836.300 | INFO | prefect.agent - Submitting flow run 'ae7d9532-0a2b-42a6-bbfb-18f3f987098a' 092841.921 | INFO | prefect.agent - Submitting flow run 'ae7d9532-0a2b-42a6-bbfb-18f3f987098a' 092846.342 | INFO | prefect.infrastructure.process - Opening process 'bouncy-dalmatian'... 092846.342 | INFO | prefect.agent - Completed submission of flow run 'ae7d9532-0a2b-42a6-bbfb-18f3f987098a' 092850.680 | ERROR | Flow run 'bouncy-dalmatian' - Flow could not be retrieved from deployment. Traceback (most recent call last): File "D:\Raghu\venv\lib\site-packages\prefect\engine.py", line 254, in retrieve_flow_then_begin_flow_run flow = await load_flow_from_flow_run(flow_run, client=client) File "D:\Raghu\venv\lib\site-packages\prefect\client.py", line 104, in with_injected_client return await fn(*args, **kwargs) File "D:\Raghu\venv\lib\site-packages\prefect\deployments.py", line 55, in load_flow_from_flow_run await storage_block.get_directory(from_path=None, local_path=".") File "D:\Raghu\venv\lib\site-packages\prefect\filesystems.py", line 100, in get_directory shutil.copytree(from_path, local_path, dirs_exist_ok=True) File "D:\Raghu\Python\lib\shutil.py", line 566, in copytree with os.scandir(src) as itr: FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\HOME\\AppData\\Local\\Temp\\tmpl2td_44jprefect' 092851.663 | INFO | prefect.infrastructure.process - Process 'bouncy-dalmatian' exited cleanly.
1
The program:
Copy code
@task
def print_hello():
    print("Hello Now!")

@flow
def print_pipeline_new():
    print_hello()

deployment = Deployment.build_from_flow(flow=print_pipeline_new,\
                                        name="Pipeline Deployment", version="1", tags=["tag"],\
                                        schedule={'interval': 120, 'timezone': "Asia/Kolkata"},
                                        work_queue_name="new-work-queue")

deployment.apply()
Version: 2.2.0 API version: 0.8.0 Python version: 3.9.13 Git commit: e3651362 Built: Tue, Aug 23, 2022 2:18 PM OS/Arch: win32/AMD64 Profile: ******** (masked) Server type: hosted
It ran one time printing "Hello Now!" and subsequent scheduled runs fail with the same error. Any help would be appreciated! Thanks.
a

Anna Geller

08/28/2022, 12:29 PM
it looks like a path issue when building deployment from Python - I believe this is fixed in the main branch and will be released next week - until now, can you try using CLI to build and apply deployment? This post explains how https://medium.com/the-prefect-blog/declarative-dataflow-deployments-with-prefect-make-ci-cd-a-breeze-fe77bdbb58d4
🙌 1
v

Venkat Ramakrishnan

08/28/2022, 4:42 PM
Thank you so much for the update! Any chances of fix being available by 1st or 2nd? I have a deadline on 5th. If it won't be available by 2nd, I will go for the CLI option. Thanks!
a

Anna Geller

08/28/2022, 6:31 PM
no guarantee on when the release will happen, and I would personally recommend the CLI approach anyway because I believe it provides a better UX keep us posted if the CLI fixed the issue for you
v

Venkat Ramakrishnan

09/01/2022, 12:33 PM
Got Prefect 2.3.0 just now, hopefully the python path problem is fixed in it. Got to try. Before that, I have this question:
Copy code
deployment = Deployment.build_from_flow(flow_name=workflow_orchestration,\
                                        name="Workflow Orchestration", \
                                        version="1", tags=["Iris"],\
                                        schedule={'interval': 600, 'timezone': "Asia/Kolkata"})
I want to provide schedules like every hour, once in a week, once in a month, etc. In CLI, I saw something like --rrule option for frequency-based scheduling. What is the equivalent option in Python? I searched online, but I could not find a comprehensive documentation that explains with examples. Thanks!
v

Venkat Ramakrishnan

09/02/2022, 2:46 AM
By the way, Prefect 2.3.0 didn't fix the issue of agent traceback..... (venv) D:\Raghu\gProject>prefect agent start -q hour-work-queue Starting v2.3.0 agent connected to http://127.0.0.1:4200/api... _ _ _ _ _ _ _____ _ _ _ _ _ _____ | _ \ _ \ __| __| __/ _| _| /_\ / __| _| \| | _| | _/ / _|| _|| | (_ | | / _ \ (_ | _|| .` | | | |_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_| Agent started! Looking for work from queue(s): hour-work-queue... 001532.074 | ERROR | prefect.agent - Traceback (most recent call last): File "D:\Raghu\Python\lib\asyncio\windows_events.py", line 462, in finish_recv return ov.getresult() OSError: [WinError 64] The specified network name is no longer available During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\Raghu\Python\lib\asyncio\proactor_events.py", line 280, in _loop_reading data = fut.result() File "D:\Raghu\Python\lib\asyncio\windows_events.py", line 817, in _poll value = callback(transferred, key, ov) File "D:\Raghu\Python\lib\asyncio\windows_events.py", line 466, in finish_recv raise ConnectionResetError(*exc.args) ConnectionResetError: [WinError 64] The specified network name is no longer available The above exception was the direct cause of the following exception: Traceback (most recent call last): File "D:\Raghu\venv\lib\site-packages\httpcore\_exceptions.py", line 8, in map_exceptions yield File "D:\Raghu\venv\lib\site-packages\httpcore\backends\asyncio.py", line 33, in read return await self._stream.receive(max_bytes=max_bytes) File "D:\Raghu\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 1274, in receive raise self._protocol.exception anyio.BrokenResourceError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\Raghu\venv\lib\site-packages\httpx\_transports\default.py", line 60, in map_httpcore_exceptions yield File "D:\Raghu\venv\lib\site-packages\httpx\_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "D:\Raghu\venv\lib\site-packages\httpcore\_async\connection_pool.py", line 253, in handle_async_request raise exc File "D:\Raghu\venv\lib\site-packages\httpcore\_async\connection_pool.py", line 237, in handle_async_request response = await connection.handle_async_request(request) File "D:\Raghu\venv\lib\site-packages\httpcore\_async\connection.py", line 90, in handle_async_request return await self._connection.handle_async_request(request) File "D:\Raghu\venv\lib\site-packages\httpcore\_async\http11.py", line 105, in handle_async_request raise exc File "D:\Raghu\venv\lib\site-packages\httpcore\_async\http11.py", line 84, in handle_async_request ) = await self._receive_response_headers(**kwargs) File "D:\Raghu\venv\lib\site-packages\httpcore\_async\http11.py", line 148, in _receive_response_headers event = await self._receive_event(timeout=timeout) File "D:\Raghu\venv\lib\site-packages\httpcore\_async\http11.py", line 177, in _receive_event data = await self._network_stream.read( File "D:\Raghu\venv\lib\site-packages\httpcore\backends\asyncio.py", line 35, in read return b"" File "D:\Raghu\Python\lib\contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "D:\Raghu\venv\lib\site-packages\httpcore\_exceptions.py", line 12, in map_exceptions raise to_exc(exc) httpcore.ReadError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "D:\Raghu\venv\lib\site-packages\prefect\agent.py", line 129, in get_and_submit_flow_runs queue_runs = await self.client.get_runs_in_work_queue( File "D:\Raghu\venv\lib\site-packages\prefect\client.py", line 918, in get_runs_in_work_queue response = await self._client.post( File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1842, in post return await self.request( File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1527, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "D:\Raghu\venv\lib\site-packages\prefect\client.py", line 257, in send await super().send(*args, **kwargs) File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1614, in send response = await self._send_handling_auth( File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1642, in _send_handling_auth response = await self._send_handling_redirects( File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1679, in _send_handling_redirects response = await self._send_single_request(request) File "D:\Raghu\venv\lib\site-packages\httpx\_client.py", line 1716, in _send_single_request response = await transport.handle_async_request(request) File "D:\Raghu\venv\lib\site-packages\httpx\_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "D:\Raghu\Python\lib\contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "D:\Raghu\venv\lib\site-packages\httpx\_transports\default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ReadError 001537.466 | INFO | prefect.agent - Submitting flow run '48ede090-c668-444e-8e67-1cb4a9837101' 001541.737 | INFO | prefect.infrastructure.process - Opening process 'natural-seagull'... 001541.811 | INFO | prefect.agent - Completed submission of flow run '48ede090-c668-444e-8e67-1cb4a9837101' 001546.709 | INFO | prefect.engine - Created flow run 'hopping-parrot' for flow 'print-pipeline-hour' 001546.836 | ERROR | Flow run 'hopping-parrot' - Crash detected! Execution was interrupted by an unexpected exception. 001546.898 | ERROR | Flow run 'natural-seagull' - Flow could not be retrieved from deployment.
This specific error makes me curious: "OSError: [WinError 64] The specified network name is no longer available" Where is it looking so that I can see if that network name is not available? What is the network name that it has problem with? Thanks.
a

Anna Geller

09/02/2022, 12:22 PM
Hi, I'm OOO. Could you open a GitHub issue showing the Prefect version you used and build a minimal example that results in this error? Thanks in advance
1
v

Venkat Ramakrishnan

09/03/2022, 8:49 AM
3 Views