Hi, I try to build a deployment and upload to minio storage(remote-file-system), the package s3fs al...
e

eli yosef

over 2 years ago
Hi, I try to build a deployment and upload to minio storage(remote-file-system), the package s3fs already install and i put this lines to head of flow.py
from prefect.filesystems import RemoteFileSystem
remote_file_system_block = RemoteFileSystem.load("minio-s3")
I get the error:
root@fd9f23f7b06c:~/flows# prefect deployment build rest_flow_3.py:websites2 -n eli -q check_url_status -sb remote-file-system/minio-s3
Found flow 'test url check flow'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 230, in coroutine_wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/prefect/utilities/asyncutils.py", line 181, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/usr/local/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/usr/local/lib/python3.8/site-packages/prefect/cli/deployment.py", line 988, in build
    deployment = await Deployment.build_from_flow(
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 755, in build_from_flow
    await deployment.upload_to_storage(ignore_file=ignore_file)
  File "/usr/local/lib/python3.8/site-packages/prefect/deployments.py", line 600, in upload_to_storage
    file_count = await self.storage.put_directory(
  File "/usr/local/lib/python3.8/site-packages/prefect/filesystems.py", line 368, in put_directory
    self.filesystem.put_file(f, fpath, overwrite=True)
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 114, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 99, in sync
    raise return_result
  File "/usr/local/lib/python3.8/site-packages/fsspec/asyn.py", line 54, in _runner
    result[0] = await coro
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 1101, in _put_file
    await self._call_s3(
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 332, in _call_s3
    await self.set_session()
  File "/usr/local/lib/python3.8/site-packages/s3fs/core.py", line 492, in set_session
    self.session = aiobotocore.session.AioSession(**self.kwargs)
TypeError: __init__() got an unexpected keyword argument 'url'
An exception occurred.
my setting is:
{  "url": "<https://172.23.0.5:900>",  "accessKey": "!!!!!!!!!!!!!!!!!",  "secretKey": "===============",  "api": "s3v4",  "path": "auto"}
1
Does the latest prefect version have any issues for others? My pipelines are all failing about a pre...
s

Samuel Hinton

about 1 year ago
Does the latest prefect version have any issues for others? My pipelines are all failing about a prefect dependency, griffe, not having a dataclasses module.
Traceback (most recent call last):
  File "/builds/arenko-group/infra/data-account/scripts/create_resources.py", line 6, in <module>
    from prefect.blocks.system import Secret
  File "/venv/lib/python3.12/site-packages/prefect/__init__.py", line 30, in <module>
    from prefect.deployments import deploy
  File "/venv/lib/python3.12/site-packages/prefect/deployments/__init__.py", line 1, in <module>
    import prefect.deployments.base
  File "/venv/lib/python3.12/site-packages/prefect/deployments/base.py", line 22, in <module>
    from prefect.client.schemas.objects import MinimalDeploymentSchedule
  File "/venv/lib/python3.12/site-packages/prefect/client/schemas/__init__.py", line 4, in <module>
    from .actions import BlockTypeUpdate, StateCreate
  File "/venv/lib/python3.12/site-packages/prefect/client/schemas/actions.py", line 7, in <module>
    from prefect._internal.compatibility.deprecated import DeprecatedInfraOverridesField
  File "/venv/lib/python3.12/site-packages/prefect/_internal/compatibility/deprecated.py", line 28, in <module>
    from prefect.utilities.callables import get_call_parameters
  File "/venv/lib/python3.12/site-packages/prefect/utilities/callables.py", line 27, in <module>
    from griffe.dataclasses import Docstring
ModuleNotFoundError: No module named 'griffe.dataclasses'