Hi all! This might be a beginner issue, but is it possible to separate common functionalities to other python modules and use them in all the flows in my project by importing the required functions?
I tried to do this, but getting
ModuleNotFoundError.
I have tried using setting
LocalRun(working_dir=)
config too, but no success. How to handle such imports?
Here's how the project is structured:
Copy code
project
|_lib
|_ util.py
|_ common.py
|_flows
|_ flow1.py // imports from util etc here which is throwing error
|_ flow2.py
|_ flow3.py
a
Anna Geller
12/22/2021, 2:26 PM
yes. You have several options to go about it:
1. Build a package, install it in your agent and this way make those modules importable
2. Add the path to your custom code to the Python PATHon the agent
3. Add a path to the agent when starting it e.g prefect agent local start -p /Users/you/path/to/your/module
4. Build a Docker storage
👍 1
Anna Geller
12/22/2021, 2:27 PM
#3 is probably the easiest option
d
dev
12/22/2021, 2:49 PM
@Anna Geller I have tried with #3, but still throwing the same error.
p
Prasanth Kothuri
12/22/2021, 3:33 PM
we organize our code this way, we set PYTHONPATH while registering the flow and also these libs available to the agent and set PYTHONPATH on the agent .. all works as expected
👍 1
upvote 2
a
Anna Geller
12/22/2021, 3:39 PM
@dev I think you can either try again and see if something is missing in your configuration, or try one of the other options I listed above
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.