https://prefect.io
Join Slack
Is `prefect-docker` supposed to be backward compatible? It doesn't seem to work anymore with `prefec...
# ask-community
l
Is 
prefect-docker
supposed to be backward compatible? It doesn't seem to work anymore with 
prefect==2.X.X
We've suddenly gotten build failures on all flows due to this https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-docker/pyproject.toml#L26 According to the README.md - it is designed to work for prefect 2.0 but this is no longer true
✅ 1
n
the version of integrations on 
main
are for 3.x the ones on 
2.x
are for 2.x versions
my most recent message in this channel may be helpful
l
seems like 
prefect-docker==0.5.3
works
I don't see 
2.x
or 
3.x
versions for that package
(I could be misunderstanding you tho)
n
are you intending to use prefect 2.x or 3.x?
l
2.x
n
https://pypi.org/project/prefect-docker/#history 
pip install 'prefect[docker]<3'
should give you what you want but basically all integrations got bumped by a minor verison (which really corresponds to a major version but its sub 1.x.y) in terms of the version numbers, 
prefect-docker<0.6.0
will be 2.x
🙌 1
l
thank you
m
Hey @Nate and @Leon Kozlowski, How did you get this working? I tried to use both 
prefect-docker<0.6.0
and 
prefect-docker==0.5.3
but both methods gave me this error: Thanks in advance for some help :)
Copy code
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker==0.5.3
    image_name: '{{ $REGISTRY }}/{{ $HARBOR_PROJECT }}/{{ $IMAGE_NAME }}'
    tag: "{{ $IMAGE_TAG }}"
    dockerfile: auto
Copy code
Running deployment build steps...
 > Running build_docker_image step...
Traceback (most recent call last):
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 154, in run_steps
    step_output = await run_step(step, upstream_outputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 124, in run_step
    step_func = _get_function_for_step(fqn, requires=keywords.get("requires"))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 73, in _get_function_for_step
    import_module(_strip_version(package).replace("-", "_"))
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect_docker/__init__.py", line 2, in <module>
    from .host import DockerHost  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect_docker/host.py", line 30, in <module>
    class DockerHost(Block):
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/main.py", line 282, in __new__
    cls = super().__new__(mcs, name, bases, new_namespace, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen abc>", line 106, in __new__
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/utilities/dispatch.py", line 100, in _register_subclass_of_base_type
    register_type(cls)
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/utilities/dispatch.py", line 154, in register_type
    key = get_dispatch_key(cls)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/utilities/dispatch.py", line 77, in get_dispatch_key
    dispatch_key = dispatch_key()
                   ^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/blocks/core.py", line 319, in __dispatch_key__
    return block_schema_to_key(cls._to_block_schema())
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/blocks/core.py", line 491, in _to_block_schema
    fields = cls.schema()
             ^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/main.py", line 664, in schema
    s = model_schema(cls, by_alias=by_alias, ref_template=ref_template)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 188, in model_schema
    m_schema, m_definitions, nested_models = model_process_schema(
                                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 581, in model_process_schema
    m_schema, m_definitions, nested_models = model_type_schema(
                                             ^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 622, in model_type_schema
    f_schema, f_definitions, f_nested_models = field_schema(
                                               ^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 248, in field_schema
    s, schema_overrides = get_field_info_schema(field)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 216, in get_field_info_schema
    schema_['default'] = encode_default(field.default)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/schema.py", line 995, in encode_default
    return pydantic_encoder(dft)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/pydantic/v1/json.py", line 90, in pydantic_encoder
    raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
TypeError: Object of type 'FieldInfo' is not JSON serializable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 42, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 311, in coroutine_wrapper
    return call()
           ^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 432, in __call__
    return self.result()
           ^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 318, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 179, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 389, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/cli/deploy.py", line 429, in deploy
    await _run_single_deploy(
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/client/utilities.py", line 100, in with_injected_client
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/cli/deploy.py", line 655, in _run_single_deploy
    await run_steps(build_steps, step_outputs, print_function=app.console.print)
  File "/opt/actions-runner/_work/_tool/Python/3.11.9/x64/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 182, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running prefect_docker.deployments.steps.build_docker_image
An exception occurred.
solved it by adding it to the requirements.txt instead of only using it in the prefect.yaml
l
I only have 
prefect-docker
in my 
prefect.yaml
❤️ 1
m
hmm then i don't know why it worked now and not a few minutes ago hahah
v
Also be careful with 
prefect worker start --pool "docker-pool" --type "docker"
command because if *--install-polic*y set to 
always
 - the latest 
prefect-docker
and 
prefect 3.0.0
will be installed ignoring your previous settings, which would logically lead to an error
upvote 2
10 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by