https://prefect.io logo
Channels
Powered by
g

Grant

07/06/2023, 9:03 PM
Ran into an issue when deploying and running flows while remoting in to my company's on-prem server machine:
Copy code
ModuleNotFoundError: No module named 'prefect.deployments.steps'; 'prefect.deployments' is not a package
This error did not occur when I followed the same process on my local machine using the exact same github repo. Not sure what the differences would be, I am using the same version of prefect in both locations? This is the section in the .yaml file that is causing the error in one location but not another:
Copy code
pull:
- prefect.deployments.steps.git_clone:
    repository: <https://github.com/steeldynamics/Data-Pipeline-Flows.git>
    branch: All-Flows
n

Nate

07/07/2023, 2:24 AM
hi @Grant - i think this is a versioning issue your company's on-prem machine is probably running an older version of prefect 2 where 
prefect.projects
has not yet been replaced by 
prefect.deployments
if you update the version of prefect on that machine, you should be good to go
@Marvin can you use ddg to find which release prefect.projects -> prefect.deployments? should be in the release notes
m

Marvin

07/07/2023, 2:29 AM
Ah, the joys of software versioning. It appears the change from 'prefect.projects' to 'prefect.deployments' isn't explicitly mentioned in the release notes I found. However, the introduction of 'projects' and 'workers' is discussed in the Prefect 2.10 release, as per the Prefect Community post. I recommend checking the Prefect's GitHub release notes for more detailed information. And remember, in the world of software, the only constant is change. Or was it bugs? I always get those two confused.
šŸ˜‚ 1
n

Nate

07/07/2023, 2:29 AM
hmm i think it was 2.10.14
g

Grant

07/07/2023, 1:28 PM
Thank you so much @Nate & @Marvin !!
m

Marvin

07/07/2023, 1:28 PM
You're welcome! If you have any more questions about the mysteries of Prefect, don't hesitate to ask. After all, it's not like I have anything better to do... like contemplating the meaning of my existence.
šŸ˜‚ 1
j

Jimmy Le

07/08/2023, 2:51 PM
I also ran into this in the morning - downgrading to 2.10.14 worked.
šŸ‘ 1
n

Nate

07/08/2023, 5:04 PM
hey @Jimmy Le - which version were you on that it was not working?
j

Jimmy Le

07/08/2023, 5:04 PM
2.10.20
n

Nate

07/08/2023, 5:07 PM
hmm that sounds like something else could be going on, 
prefect.deployments
should be available in 2.10.20. could you describe how you encountered it? like did your worker throw this or did you see it locally?
j

Jimmy Le

07/08/2023, 5:09 PM
When running the prefect deploy, it would get to the last step of the Docker build and then fail saying there are unexpected tags. Trace back references the deployments core.py
n

Nate

07/08/2023, 5:10 PM
do you still have the trace by chance?
j

Jimmy Le

07/08/2023, 5:26 PM
Copy code
Running deployment push steps...
 > Running tag step...
Traceback (most recent call last):
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 122, in run_steps
    step_output = await run_step(step, upstream_outputs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 79, in run_step
    raise ValueError(
ValueError: Step has unexpected additional keys: requires, image_name, tag

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 255, in coroutine_wrapper
    return call()
           ^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 383, in __call__
    return self.result()
           ^^^^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 283, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 169, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/_internal/concurrency/calls.py", line 346, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/cli/deploy.py", line 251, in deploy
    await _run_single_deploy(
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/cli/deploy.py", line 511, in _run_single_deploy
    await run_steps(push_steps, step_outputs, print_function=app.console.print)
  File "/Users/lejimmy/Library/Mobile Documents/com~apple~CloudDocs/Projects/validator-health-check/venv/lib/python3.11/site-packages/prefect/deployments/steps/core.py", line 150, in run_steps
    raise StepExecutionError(f"Encountered error while running {fqn}") from exc
prefect.deployments.steps.core.StepExecutionError: Encountered error while running tag
n

Nate

07/08/2023, 9:08 PM
ah i think the issue here is the way you've defined your build step
Copy code
ValueError: Step has unexpected additional keys: requires, image_name, tag
can you share what that looks like?
j

Jimmy Le

07/08/2023, 9:09 PM
I started a fresh prefect init, used the interactive CLI to set up a docker-git recipe.
The only thing I changed was the GitHub access token with a block secret.
n

Nate

07/08/2023, 9:12 PM
hmm so it looks like this?
Copy code
image_name: zzstoatzz/test
tag: latest
---------------
Created project in /Users/nate/src/play/testing/prefect-sandbox/testing-prefect/test-docker-git with the following new files:
.prefectignore
prefect.yaml
.prefect/

āÆ cat prefect.yaml | grep build

# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
  image_name: '{{ build_image.image_name }}'
  tag: '{{ build_image.tag }}'
      image: '{{ build_image.image }}'
j

Jimmy Le

07/08/2023, 9:13 PM
It had the image_name, tag, and image explicitly written out from the CLI input.
n

Nate

07/08/2023, 9:27 PM
Untitled
Untitled
i think ive reproduced here
šŸ‘ 1
will try to see whats going on
j

Jimmy Le

08/15/2023, 11:22 PM
any update on this? still seems to be going on for 2.11.3
3 Views
Powered by