Hi everyone! I am trying to understand how to reference a custom module in my flow code deployed on an Azure ACI worker. I am using a git clone step to get the flow code onto the worker, whenever I run the flow and reference the utils package I receive a no module utils found. I thought about adding a python path but I was wondering if that's the correct approach? Thanks!!! i.e. folder structure 📁 prefect-project/ ├── 📁 flows/ │ └── 📁 operations/ │ ├── test-communications.py │ └── other_flow.py ├── 📁 src/ │ └── 📁 utils/ │ ├── init.py │ ├── slack.py │ └── snowflake.py ├── prefect.yaml ├── pyproject.toml ├── requirements.txt └── README.md

hi @Keith Smith - there's an example repo that might be helpful for you here since you're using ACI, I would recommend installing your custom module into an

image

that you can then reference on your work pool / deployments

Because we are early days in the setup we were hoping to keep it in the code repo to make deployment easy, we don't want to separate the module from the flow yet. Using a custom docker image isn't an option either because of the ci/cd setup required being a blocker at the moment

if you can’t build an image or install it from a registry id just use run_shell_script in a pull step to install whatever / set a path @Marvin concisely show run_shell_script example in pull step of prefect.yaml

To use

run_shell_script

in a

pull

step within a

prefect.yaml

file, you can refer to the following example. This example sets up a path by running a shell script during the pull process:

yaml
pull:
  - prefect.deployments.steps.run_shell_script:
      id: setup-path
      script: |
        #!/bin/bash
        echo "Setting up path"
        export PATH=$PATH:/new/path
      stream_output: true
      expand_env_vars: true

In this setup, the

run_shell_script

step can execute a shell script, which in this example sets a new path.

expand_env_vars: true

ensures that environment variables are expanded within the script. For more detailed information, you can visit the official documentation here: Prefect Deployment Guide.

@Marvin When I add the shell script I receive a FileNotFoundError: [Errno 2] No such file or directory: '#!/bin/bash' error? The worker is an Azure ACI

It looks like there isn't a specific GitHub issue related to your

FileNotFoundError

with

run_shell_script

in the context of Azure ACI, but I found some useful details in the Prefect documentation on using Azure Container Instances (ACI). The issue you're facing might be related to the environment in which your script is executed. Azure Container Instances require specific permissions and setup steps to run properly. 1. Ensure Dependencies: Make sure the necessary dependencies are available in the Azure environment. For running a shell script, you'd need a shell environment that supports bash or sh. 2. ACI Configuration: It's important to configure ACI correctly with all required permissions. Ensure your setup aligns with the Azure ACI Worker Guide. 3. Validate Environment: Since you are dealing with a scripting error, ensure your script runs locally in a similar environment to quickly validate any potential issues. 4. Execution Command: Ensure the script is executed with the correct shell interpreter (e.g.,

/bin/bash

or

/bin/sh

). The environment might not have the expected shell by default. By verifying these points, you can ensure a smoother execution of your shell script in Azure ACI with Prefect. Hopefully, this helps you get back to conquering the universe... or at least running your Prefect flow successfully.