hey once again when executing a

LocalRun

with

LocalExecutor

how can I do the equivalent of

sys.path.append

to the run config working dir currently flows are working without a backend but then when I start to run on my server I get a an error with

ModuleNotFound

currently my structure is as such

MainProject/
|-coreLogicModule/
|-PrefectStack/
    flows/    -need access to coreLogic
    runflows.py

You can set the LocalRun working dir. Example here . And then you can maybe do

sys.path.append

inside a task if you still need it?

hey @Kevin Kho - not quite following… just updated my question maybe you could recommend a better dir structure ?

@Mike Lev the easiest way to solve it would be to: • add setup.py at the “MainProject” level, • this setup.py would reference the

coreLogicModule

• to run this, you

cd

into the MainProject dir and run:

pip install -e .

• as long as this package is installed in the environment from which you start the agent, and from the environment from which you register your flows, this should work without modifying the path or even adding the path to the agent • here is an example folder structure and setup.py: https://github.com/anna-geller/packaging-prefect-flows here is an example setup.py that you could use:

from setuptools import setup, find_packages

with open('requirements.txt') as f:
    requirements = f.read().splitlines()

setup(
    name="coreLogicModule",
    version='0.01', # version this as you like
    packages=find_packages(),
    install_requires=requirements
)

LMK if you have questions to this.

Anna’s suggestion is definitely more proper but I was thinking that if you did:

import sys
sys.path.append("../..")

import thing_in_directory_above

@task
def abc():
   return thing_in_directory_above()

with Flow(...) as flow:
    abc()

this will not work because the import happens during “build” time when the flow is registered, but you want it to be deferred and happen during run time when the flow is executed. The way to do this would be to use a task to defer appending the path. For example:

@task
def abc():
   import sys
   sys.path.append("../..")
   import thing_in_directory_above
   return thing_in_directory_above()

with Flow(...) as flow:
    abc()

and now the appending is done when the Flow is running.

amazing ... its working slowly also ... any straightforward approaches to config serialization behavior for python objs and numpy arrays so I dont get the obnoxious json dumps errors?

Think of it this way, everything that you are transmitting to Prefect happens through an API so it has to be JSONSerializeable. So you can’t pass stuff like Pandas DataFrames or Numpy Arrays through any API.