Hi! I am working on setting up a simple CI/CD process that registers new flows each time a push is made to main. My folder structure is inspired by the gitlab data team so at the root of my project i have a folder orchestrate. Within that folder i have a folder tasks - within this there are subdirectories that contain custom modules that store tasks we use throughout our flows. While registering flows via the GitHub action, the Prefect CI is complaining about not being able to find a module in the tasks folder. When I run this locally, I do not get any issues. I am pretty sure it is related to my PYTHONPATH setup within the GitHub flow but I have not had any luck troubleshooting. Has anyone out there run into a similar issue?

Yes, the fix for this is mentioned here One straightforward way for you to fix this is either to add the module to PYTHONPATH or add the following code to the beginning of the script:

import sys
sys.path.append("path_to_your_module")

What script am I adding this to? Here is my current job:

jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.7'
          cache: 'pip'

      - name: Install Prefect
        run: |
          cd $GITHUB_WORKSPACE
          pip install -r requirements.txt
          prefect register -p ./orchestrate/flows/authentics/database_loader.py --project $PREFECT_PROJECT

That would be inside the Python file, but I think you are right that you can just add to PYTHONPATH. Can’t you add it after installing the requirements.txt?

I've tried that but the documentation I have found related to that within GitHub workflows is not very clear. I may shift and move towards using a python script to register instead of the CLI

we have an entire Discourse tag for that https://discourse.prefect.io/tag/ci-cd including a blog post from Jean for Prefect 1.0 https://www.prefect.io/guide/blog/deploying-prefect-flows-with-github-actions/

thanks!

Ah I see