https://prefect.io logo
Title
b

Ben Muller

11/04/2022, 8:13 AM
Hey Prefect, I am completely lost on this one, I am trying to deploy a flow from a docker container and getting this error 🧵
Traceback (most recent call last):
  File "//deploy/scripts/deploy_flows.py", line 31, in <module>
    raise Exception(process.stderr)
Exception: Traceback (most recent call last):
  File "//./flows/betfair/deployments.py", line 19, in <module>
    strategy_account.deploy()
  File "/src/prefect_util/prefect_deployment.py", line 52, in deploy
    self.deployment.apply(upload=True)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 201, in coroutine_wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 152, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/usr/local/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/usr/local/lib/python3.9/site-packages/prefect/deployments.py", line 608, in apply
    await self.upload_to_storage()
  File "/usr/local/lib/python3.9/site-packages/prefect/deployments.py", line 572, in upload_to_storage
    file_count = await self.storage.put_directory(
  File "/usr/local/lib/python3.9/site-packages/prefect/filesystems.py", line 481, in put_directory
    return await self.filesystem.put_directory(
  File "/usr/local/lib/python3.9/site-packages/prefect/filesystems.py", line 338, in put_directory
    included_files = filter_files(
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/filesystem.py", line 42, in filter_files
    ignored_files = {p.path for p in spec.match_tree_entries(root)}
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/filesystem.py", line 42, in <setcomp>
    ignored_files = {p.path for p in spec.match_tree_entries(root)}
  File "/usr/local/lib/python3.9/site-packages/pathspec/pathspec.py", line 232, in match_tree_entries
    yield from self.match_entries(entries)
  File "/usr/local/lib/python3.9/site-packages/pathspec/pathspec.py", line 148, in match_entries
    for entry in entries:
  File "/usr/local/lib/python3.9/site-packages/pathspec/util.py", line 155, in iter_tree_entries
    yield from _iter_tree_entries_next(os.path.abspath(root), '', {}, on_error, follow_links)
  File "/usr/local/lib/python3.9/site-packages/pathspec/util.py", line 225, in _iter_tree_entries_next
    yield from _iter_tree_entries_next(root_full, node_rel, memo, on_error, follow_links)
  File "/usr/local/lib/python3.9/site-packages/pathspec/util.py", line 225, in _iter_tree_entries_next
    yield from _iter_tree_entries_next(root_full, node_rel, memo, on_error, follow_links)
  File "/usr/local/lib/python3.9/site-packages/pathspec/util.py", line 225, in _iter_tree_entries_next
    yield from _iter_tree_entries_next(root_full, node_rel, memo, on_error, follow_links)
  File "/usr/local/lib/python3.9/site-packages/pathspec/util.py", line 194, in _iter_tree_entries_next
    raise RecursionError(real_path=dir_real, first_path=memo[dir_real], second_path=dir_rel)
pathspec.util.RecursionError: ('/', '', 'dev/fd/7')
for the record - if I run this locally it does not fail, for some reason it only fails in the docker container 🤷
k

Khuyen Tran

11/04/2022, 3:35 PM
What does your deployment.yaml look like?
b

Ben Muller

11/04/2022, 9:14 PM
It's deployed via the python api. I can't see that producing a yaml file?
Maybe it's a s3fs issue?