https://prefect.io logo
Join Slack
Powered by
Hey all! Can anybody help with a github storage b...
# prefect-community
z
Hey all! Can anybody help with a github storage block? When I run the flow, I keep getting the message 
Flow could not be retrieved from deployment
with a stack trace that leads to a 
NotImplementedError
. I have tried several variations on the --path argument, tried a public repo instead of a private - nothing works! Please help! flow-code:
Copy code
import sys
import prefect
from prefect import flow, task, get_run_logger
from utilities import AN_IMPORTED_MESSAGE


@task
def log_task(name):
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello %s!", name)
    <http://logger.info|logger.info>("Prefect Version = %s 🚀", prefect.__version__)
    logger.debug(AN_IMPORTED_MESSAGE)


@flow()
def log_flow(name: str):
    log_task(name)


if __name__ == "__main__":
    name = sys.argv[0]
    log_flow(name)
deployment CLI command:
Copy code
prefect deployment build ./src/log_flow.py:log_flow -n log-simple-test -q test -sb github-repository/test-public-repo --apply
log_flow-deployment.yaml:
Copy code
###
### A complete description of a Prefect Deployment for flow 'log-flow'
###
name: log-simple-test
description: null
version: 0309032a075c5be33c8c074988fd6e41
# The work queue that will handle this deployment's runs
work_queue_name: test
work_pool_name: default-agent-pool
tags: []
parameters: {}
schedule: null
is_schedule_active: true
infra_overrides: {}

###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: log-flow
manifest_path: null
infrastructure:
  type: process
  env: {}
  labels: {}
  name: null
  command: null
  stream_output: true
  working_dir: null
  _block_document_id: ce44b1b5-b0a4-4034-a724-b4660da17555
  _block_document_name: anonymous-fa985916-1aed-4f2b-89f2-5f5e2c278ae0
  _is_anonymous: true
  block_type_slug: process
  _block_type_slug: process
storage:
  repository_url: <https://github.com/zachloertscher/test-public-repo>
  reference: main
  credentials: null
  _block_document_id: a2fbc663-3d11-4551-9350-49eec1a3de54
  _block_document_name: test-public-github-repo
  _is_anonymous: false
  block_type_slug: github-repository
  _block_type_slug: github-repository
path: /Users/zachloertscher/test-public-github-repo
entrypoint: src\log_flow.py:log_flow
parameter_openapi_schema:
  title: Parameters
  type: object
  properties:
    name:
      title: name
      position: 0
      type: string
  required:
  - name
  definitions: null
timestamp: '2023-04-04T13:57:58.625897+00:00'
Stack trace:
Copy code
09:58:36.212 | INFO    | prefect.agent - Submitting flow run '0b7f9e6d-bbcd-4b21-8c41-12d735d31816'
09:58:36.874 | INFO    | prefect.infrastructure.process - Opening process 'dark-curassow'...
09:58:36.978 | INFO    | prefect.agent - Completed submission of flow run '0b7f9e6d-bbcd-4b21-8c41-12d735d31816'
C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
09:58:41.105 | INFO    | Flow run 'dark-curassow' - Downloading flow code from storage at '/Users/zachloertscher/test-public-github-repo'
09:58:41.113 | ERROR   | Flow run 'dark-curassow' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 277, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\deployments.py", line 194, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect_github\repository.py", line 146, in get_directory
    process = await run_process(cmd, stream_output=(out_stream, err_stream))
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 258, in run_process
    async with open_process(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 199, in __aenter__     
    return await anext(self.gen)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 200, in open_process
    process = await _open_anyio_process(command, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 162, in _open_anyio_process
    process = await asyncio.create_subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1634, in subprocess_shell
    transport = await self._make_subprocess_transport(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
09:58:41.292 | ERROR   | prefect.engine - Engine execution of flow run '0b7f9e6d-bbcd-4b21-8c41-12d735d31816' exited with unexpected exception
Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 2131, in <module>
    enter_flow_run_engine_from_subprocess(flow_run_id)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 202, in enter_flow_run_engine_from_subprocess
    ).result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\schemas.py", line 107, in result
    return get_state_result(self, raise_on_failure=raise_on_failure, fetch=fetch)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\states.py", line 76, in get_state_result
    return _get_state_result(state, raise_on_failure=raise_on_failure)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 260, in coroutine_wrapper
    return call()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 245, in __call__
    return self.result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 173, in result
    return self.future.result(timeout=timeout)
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 391, in __get_result
    raise self._exception
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\call    transport = await self._make_subprocess_transport(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport    raise NotImplementedError
NotImplementedError
09:58:41.948 | ERROR   | prefect.infrastructure.process - Process 'dark-curassow' exited with status code: 1
My repo: https://github.com/zachloertscher/test-public-repo I've been at this for hours - any help would be so very much appreciated!
k
When you run your deployment command, the 
<path-to-flow.py:flow-function>
should be relative to the root of the repository. If your current directory is the root of the repo, and 
src
is a folder in that root, deploy with
Copy code
prefect deployment build src/log_flow.py:log_flow -n log-simple-test -q test -sb github-repository/test-public-repo --apply
the path in your deployment yaml is
Copy code
path: /Users/zachloertscher/test-public-github-repo
whereas the path in my deployment yaml is
Copy code
path: ''
which leads to this log
Copy code
09:58:41.105 | INFO    | Flow run 'dark-curassow' - Downloading flow code from storage at '/Users/zachloertscher/test-public-github-repo'
z
Thanks, @Kevin Grismore! The error persists though, unfortunately: deployment command: 
prefect deployment build src/log_flow.py:log_flow -n log-simple-test -q test -sb github-repository/test-public-repo --apply
deployment.yaml
Copy code
###
### A complete description of a Prefect Deployment for flow 'log-flow'
###
name: log-simple-test
description: null
version: eecfa433d451cb21450c33d154c33221
# The work queue that will handle this deployment's runs
work_queue_name: test
work_pool_name: default-agent-pool
work_pool_queue_name: null
tags: []
parameters: {}
schedule: null
infra_overrides: {}

###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: log-flow
manifest_path: null
infrastructure:
  type: process
  env: {}
  labels: {}
  name: null
  command: null
  stream_output: true
  working_dir: null
  _block_document_id: ce44b1b5-b0a4-4034-a724-b4660da17555
  _block_document_name: anonymous-fa985916-1aed-4f2b-89f2-5f5e2c278ae0
  _is_anonymous: true
  block_type_slug: process
  _block_type_slug: process
storage:
  repository_url: <https://github.com/zachloertscher/test-public-repo>
  reference: main
  credentials: null
  _block_document_id: 73fa6dd2-fbd8-488a-8671-e7724e6bf19f
  _block_document_name: test-public-repo
  _is_anonymous: false
  block_type_slug: github-repository
  _block_type_slug: github-repository
path: ''
entrypoint: src\log_flow.py:log_flow
parameter_openapi_schema:
  title: Parameters
  type: object
  properties:
    name:
      title: name
      position: 0
      type: string
  required:
  - name
  definitions: null
timestamp: '2023-04-04T14:47:50.004503+00:00'
stack trace:
Copy code
10:48:05.872 | INFO    | prefect.agent - Submitting flow run 'c40b8b48-3f96-45ab-ba74-864de63813e7'
10:48:06.779 | INFO    | prefect.infrastructure.process - Opening process 'amigurumi-lemming'...
10:48:07.029 | INFO    | prefect.agent - Completed submission of flow run 'c40b8b48-3f96-45ab-ba74-864de63813e7'
C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
10:48:10.673 | INFO    | Flow run 'amigurumi-lemming' - Downloading flow code from storage at ''
10:48:10.681 | ERROR   | Flow run 'amigurumi-lemming' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 277, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\deployments.py", line 194, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect_github\repository.py", line 146, in get_directory
    process = await run_process(cmd, stream_output=(out_stream, err_stream))
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 258, in run_process
    async with open_process(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 200, in open_process
    process = await _open_anyio_process(command, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 162, in _open_anyio_process
    process = await asyncio.create_subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1634, in subprocess_shell
    transport = await self._make_subprocess_transport(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
10:48:10.897 | ERROR   | prefect.engine - Engine execution of flow run 'c40b8b48-3f96-45ab-ba74-864de63813e7' exited with unexpected exception
Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 2131, in <module>
    enter_flow_run_engine_from_subprocess(flow_run_id)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 202, in enter_flow_run_engine_from_subprocess
    ).result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\schemas.py", line 107, in result
    return get_state_result(self, raise_on_failure=raise_on_failure, fetch=fetch)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\states.py", line 76, in get_state_result
    return _get_state_result(state, raise_on_failure=raise_on_failure)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 260, in coroutine_wrapper
    return call()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 245, in __call__
    return self.result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 173, in result
    return self.future.result(timeout=timeout)
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 391, in __get_result
    raise self._exception
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 218, in _run_async
    result = await coro
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\states.py", line 91, in _get_state_result
    raise await get_state_exception(state)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 277, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\deployments.py", line 194, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect_github\repository.py", line 146, in get_directory
    process = await run_process(cmd, stream_output=(out_stream, err_stream))
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 258, in run_process
    async with open_process(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 200, in open_process
    process = await _open_anyio_process(command, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\processutils.py", line 162, in _open_anyio_process
    process = await asyncio.create_subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1634, in subprocess_shell
    transport = await self._make_subprocess_transport(
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
10:48:11.516 | ERROR   | prefect.infrastructure.process - Process 'amigurumi-lemming' exited with status code: 1
Does running asyncio on windows cause any issues like this
k
Sorry, I haven't tried that myself
z
Have you had success with github-repository storage bocks in the past?
k
I've been using both gitlab and bitbucket repo blocks
Based on the logs it looks like you're running the agent locally on windows?
z
Yes it's just a locally run agent on windows
k
I did find this based on the error which seems to be about running 
asyncio
on Windows: https://stackoverflow.com/questions/44633458/why-am-i-getting-notimplementederror-with-async-and-await-on-windows
z
Maybe I'll give it a try with 
wsl
- thanks again for your help!
k
np!
⭐ 1
gratitude thank you 1
z
Just to circle back on this - in case anybody is wondering, I tried running my prefect agent on WSL (windows subsystem for linux) and it worked fine Would be awesome to have agents working on local windows machines for github-repo storage blocks!
👀 1
b
Hey Zachary, I appreciate you sharing the solution that you discovered. Just out of curiosity, do you have prefect-dask installed in your exection environment? We've previously seen the 
NotImplementedErrror
thrown when that package was installed.
z
Hi Bianca - yeah I did prefect-dask installed in my environment
b
Ahh I see. That may have contributed to the error message then.
z
Yes - I believe so! I went ahead and ran 
pip uninstall prefect-dask
to try it on Windows again, but it threw a FileNotFound error (& still works on WSL). Should I submit a github issue?:
Copy code
Agent started! Looking for work from queue(s): test...
15:12:08.917 | INFO    | prefect.agent - Submitting flow run '53430697-be66-43b2-ab02-36ee3d216c2e'
15:12:09.879 | INFO    | prefect.infrastructure.process - Opening process 'wisteria-pillbug'...
15:12:10.107 | INFO    | prefect.agent - Completed submission of flow run '53430697-be66-43b2-ab02-36ee3d216c2e'
C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
15:12:14.341 | INFO    | Flow run 'wisteria-pillbug' - Downloading flow code from storage at ''
15:12:15.866 | ERROR   | Flow run 'wisteria-pillbug' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1016, in get_code
  File "<frozen importlib._bootstrap_external>", line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ZACHAR~1.LOE\\AppData\\Local\\Temp\\tmpljqhd9k6prefect\\containers\\prefect\\src\\flows\\log_flow.py'      

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 277, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\deployments.py", line 206, in load_flow_from_flow_run
    flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\flows.py", line 809, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\importtools.py", line 201, in import_object
    module = load_script_as_module(script_path)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\importtools.py", line 164, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'containers\\prefect\\src\\flows\\log_flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
15:12:16.333 | ERROR   | prefect.engine - Engine execution of flow run '53430697-be66-43b2-ab02-36ee3d216c2e' exited with unexpected exception
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1016, in get_code
  File "<frozen importlib._bootstrap_external>", line 1073, in get_data
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ZACHAR~1.LOE\\AppData\\Local\\Temp\\tmpljqhd9k6prefect\\containers\\prefect\\src\\flows\\log_flow.py'      

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 2131, in <module>
    enter_flow_run_engine_from_subprocess(flow_run_id)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 202, in enter_flow_run_engine_from_subprocess
    ).result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\schemas.py", line 107, in result
    return get_state_result(self, raise_on_failure=raise_on_failure, fetch=fetch)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\states.py", line 76, in get_state_result
    return _get_state_result(state, raise_on_failure=raise_on_failure)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 260, in coroutine_wrapper
    return call()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 245, in __call__
    return self.result()
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 173, in result
    return self.future.result(timeout=timeout)
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 439, in result
    return self.__get_result()
  File "C:\Users\Zachary.Loertscher\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 391, in __get_result
    raise self._exception
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\_internal\concurrency\calls.py", line 218, in _run_async
    result = await coro
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\states.py", line 91, in _get_state_result
    raise await get_state_exception(state)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\engine.py", line 277, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\client\utilities.py", line 40, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\deployments.py", line 206, in load_flow_from_flow_run
    flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 91, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\flows.py", line 809, in load_flow_from_entrypoint
    flow = import_object(entrypoint)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\importtools.py", line 201, in import_object
    module = load_script_as_module(script_path)
  File "C:\Users\Zachary.Loertscher\repos\data_warehouse_pipeline\.venv\lib\site-packages\prefect\utilities\importtools.py", line 164, in load_script_as_module
    raise ScriptError(user_exc=exc, path=path) from exc
prefect.exceptions.ScriptError: Script at 'containers\\prefect\\src\\flows\\log_flow.py' encountered an exception: FileNotFoundError(2, 'No such file or directory')
15:12:17.046 | ERROR   | prefect.infrastructure.process - Process 'wisteria-pillbug' exited with status code: 1
However, the flow functions just fine on WSL
Copy code
15:13:17.239 | INFO    | prefect.agent - Submitting flow run '98c75ed1-19cd-45d6-b314-dd17306931df'
15:13:18.450 | INFO    | prefect.infrastructure.process - Opening process 'shiny-stingray'...
15:13:18.677 | INFO    | prefect.agent - Completed submission of flow run '98c75ed1-19cd-45d6-b314-dd17306931df'
/usr/lib/python3.8/runpy.py:127: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
15:13:20.275 | INFO    | Flow run 'shiny-stingray' - Downloading flow code from storage at ''
15:13:23.490 | INFO    | Flow run 'shiny-stingray' - Created task run 'log_task-99465d2b-0' for task 'log_task'
15:13:23.491 | INFO    | Flow run 'shiny-stingray' - Executing 'log_task-99465d2b-0' immediately...
15:13:24.216 | INFO    | Task run 'log_task-99465d2b-0' - Hello Marvin!
15:13:24.216 | INFO    | Task run 'log_task-99465d2b-0' - Prefect Version = 2.7.8 🚀
15:13:24.477 | INFO    | Task run 'log_task-99465d2b-0' - Finished in state Completed()
15:13:24.763 | INFO    | Flow run 'shiny-stingray' - Finished in state Completed('All states completed.')
15:13:25.358 | INFO    | prefect.infrastructure.process - Process 'shiny-stingray' exited cleanly.
I don't know if there's another conflicting package, or some discrepency in the paths that prefect is looking for on a windows machine when running an agent locally..
2 Views
Open in Slack
PreviousNext
Powered by