https://prefect.io
Join Slack
Apologies if this has already been answered--but in the 'old' Prefect we started up agents with the ...
# ask-community
m
Apologies if this has already been answered--but in the 'old' Prefect we started up agents with the -p flag so that we can point them to local modules that our flows needed to import. This functionality seems to have gone away with Orion. Is there some analogous way to indicate to the agent the location of local files that need to be imported? For reference, the short stack trace I'm getting is 
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/tmp/flow-script-calculate-blob-size4cemrmn_.py", line 1, in <module>
ModuleNotFoundError: No module named 'local_module'
✅ 1
k
Can I see what the DeploymentSpec looks like in this case?
m
Copy code
DeploymentSpec(flow_location=/path/to/flow.py,
               name="blob-size-calc-test-deployment",
               schedule=CronSchedule(cron="30 03 * * *"),
               flow_name="calculate-blob-size",
               parameters={"st_acct": "oitdatatransferdev", "container_name": "prefect-orion-test-storage", "config_path": /path/to/configs.yml},
               tags=["test"])
k
What does this end up running on? Subprocess, Docker or Kubernetes?
m
subprocess
k
I would say: 1. The agent might be running in a directory without access to those files so you can try running it from the same folder so the subprocess will have access. 2. You can add the folder to the PYTHONPATH so it can be imported from other places. 3. Similar to two, you can make it a module and 
pip install
the module
💯 1
👍 1
m
I was hoping to avoid packaging the module...for 1 how do i determine where the agent is running?
k
Just the existing directory when you did 
prefect agent start work_queue_id
m
ah gotcha, thank you Kevin!
k
Maybe it will work. Not 100% sure. For Prefect 1, that is the case.
👍 1
m
I'll keep you posted--about to test it.
a
if you really want to make it easy, you can create a conda environment and install your custom module as an editable package inside
Copy code
pip install -e .
m
good call, thanks Anna!
👍 1
@Kevin Kho starting the agent from the directory containing the imported file worked! thank you!
👍 1
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by