Hi there, I'm trying to understand how I can deplo...
# ask-community
b
Hi there, I'm trying to understand how I can deploy flows without having flow dependencies installed in my CI/CD environment. I am baking my flow and dependencies into a docker image, and trying to register the flow metadata with Prefect Cloud, but Prefect is trying to load my flow from its entrypoint and giving this error:
Copy code
prefect.exceptions.ScriptError: Script at 'main.py' encountered an exception: ModuleNotFoundError("No module named '<flow dependency>'")
An exception occurred.
1
My flow is defined in a
prefect.yaml
file as follows:
Copy code
name: data-eng-pipelines/geocode
prefect-version: 2.14.5

build: null

push: null

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/

deployments:
- name: geocode-deployment
  version: null
  tags: []
  description: null
  schedule: {}
  flow_name: null
  entrypoint: main.py:main
  parameters: {}
  work_pool:
    name: '{{ prefect.variables.workPool }}'
    work_queue_name: null
    job_variables:
      image: '{{ prefect.variables.image }}:sample-{{ prefect.variables.tag }}'
I strongly wish to separate CI/CD from dependency install, if possible, because the goal of this project is to have various flows which have their own separately managed dependencies. Are there any recommended solutions?
k
at the moment there's no way around this that I know of. I always install the requirements for my flows in my CI runner just before deploying them
b
Fair enough, thanks for letting me know. This is a major pain point I've had since Prefect 1, and I had mistakenly thought it was resolved.
a
mind upvoting this issue? https://github.com/PrefectHQ/prefect/issues/9512 if you find it relevant
👍 1
b
Thanks for the link, @Alexander Azzam, added my $0.02.
i
This is how I get around Prefect loading unnecessary dependencies on a deploy: In my CICD pipeline: (we use CircleCI)
Copy code
- run:
    name: Deploy the Prefect Flow
    command: |
      export ENVIRONMENT='PREFECT_DEPLOY'
      prefect --no-prompt deploy --name <<parameters.environment>>
Then, in my
main.py
where the deploy does a partial against the function in the entry point to grab the metadata off the flow decorator, (pulling in all the imports) I wrap the module imports unnecessary to the deploy within a conditional so they're skipped only during deploy.
Copy code
if not getenv(
    'ENVIRONMENT'
) == 'PREFECT_DEPLOY':
    from dependent_module.app import main

@flow(
    name='best-flow-evar'
)
def run_flow():
    main()


if __name__ == "__main__":
    run_flow()