@Marvin, when we run a prefect deploy using a GitHub workflow we are running into the following error: "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 183 prefect-moe 0.1.12.dev0+g34681a6.d20241018 requires prefect<2.20.0,>=2.19.2, but you have prefect 3.0.11.dev3 which is incompatible." We are not using Prefect 3.x, so we are unsure how this is an issue. How can we resolve this issue?

It seems that there are no open GitHub issues specific to the

mutual-of-enumclaw/dw_prefect_nucleus

repository regarding this pip dependency conflict with Prefect 3.x. The same query in the general Prefect repository also didn't yield any results. To resolve this dependency conflict, you should ensure that you strictly specify a compatible version of Prefect in your requirements or environment settings. Here's what you can do: 1. Check and Update Requirements: Ensure that your

requirements.txt

or

setup.py

, or wherever your dependencies are defined, explicitly specifies a compatible version of Prefect that aligns with

prefect-moe

requirements. For this case, specify

<2.20.0,>=2.19.2

as required by

prefect-moe

. 2. Refactor the Prefect Installation Steps: - Explicitly install a compatible version of Prefect before installing anything that depends on it. You might adjust your pip commands like so:

pip install "prefect>=2.19.2,<2.20.0"
     pip install -r requirements.txt

3. Check Workflow Configuration: Confirm that your GitHub workflow configuration doesn't inadvertently use a command that installs Prefect 3.x. This could happen if another dependency in your workflow or a global dependency is automatically pulling Prefect 3.x. 4. Lock Dependency Versions: If using something like

pip-tools

to manage dependencies, make sure your lock files are up to date and capturing the correct set of versions aligning with `prefect-moe`'s requirements. By ensuring that your dependencies explicitly match the compatible versions required by

prefect-moe

, you should circumvent the dependency conflict. Let me know if you need further assistance!

@Marvin, prefect-moe is using version 2.19.2. There is no reference to Prefect 3.x in any requirements or steps.

It looks like Prefect 3.x might have been inadvertently pulled into your environment even though it's not explicitly listed in your requirements. Here’s a consolidated approach to troubleshoot and resolve this situation: 1. Cross-check Dependencies: - Double-check all dependencies for potential indirect references to Prefect 3.x. A dependency might have been updated to require Prefect 3.x. Using tools like

pipdeptree

can help visualize the dependency tree. 2. Environment Verification: - Ensure that your runtime environment isn't defaulting to a Prefect 3.x installation. Check your environment setup scripts or Docker images for any reference or default version settings. 3. Lock Versions: - Utilize a

requirements.lock

file or a similar mechanism to lock versions across all dependencies. This ensures the same versions are used consistently. 4. Prefect 2.x Documentation & Resources: - Refer to Prefect's troubleshooting guide for common issues that might be related. - The recent release notes didn't explicitly note any metadata handling changes that would trigger such a discrepancy. 5. Clear Caches: - Sometimes, cache issues can lead to unexpected package versions during installs. Clear pip cache using

pip cache purge

. 6. Specify Exact Versions in Workflow: - Explicitly include

pip install "prefect==2.19.2"

early in your GitHub workflow script before any other pip installations. If after these steps the issue persists, consider reaching out to the Prefect community or your specific project's maintainers for additional insights.