https://prefect.io logo
y

Yuri Caruso

11/20/2020, 10:47 PM
Hello I'm experiencing a problem that I don't know how to solve. I have the following program (agent.py) that starts with the supervisor when starting my Linux machine. It aims to load all the folders and files of my project where the flows are.
Copy code
from prefect.agent.local.agent import LocalAgent
from os.path import dirname, abspath, join
from os import listdir, path as p,walk

path = abspath(join(dirname(__file__)))

import_paths=[]
for file in listdir(path):
    if not file.startswith(".") and p.isdir(file) and file in ['db','service','flows','util'] :
        import_paths.append(join(path,file))

for path_import in import_paths:
    paths = [p.join(root, name)
                for root, dirs, files in walk(path_import)
                for name in dirs
                if not name.startswith(".") and not name.startswith("__")]
    import_paths.extend(paths)

LocalAgent(import_paths=import_paths,show_flow_logs=True).start()
My Supervisor File is as follows:
Copy code
[program:start_agent_prefect]
command=python3 /home/dev/projetos/prefect-tasks/agent.py
user=dev
autostart=true
autorestart=true
stderr_logfile=/var/log/agent_prefect.err.log
stdout_logfile=/var/log/agent_prefect.out.log
However, the prefect agent is unable to execute the Flows as it cannot find the prefect giving the message below:
Copy code
[Errno 2] No such file or directory: 'prefect'

[2020-11-20 22: 21: 28,139] INFO - agent | Found 1 flow run (s) to submit for execution.
[2020-11-20 22: 21: 28,176] INFO - agent | Deploying flow run 3ab508f4-731c-450b-8b30-bdaaaeba6117
[2020-11-20 22: 21: 28,180] ERROR - agent | Logging platform error for flow run 3ab508f4-731c-450b-8b30-bdaaaeba6117
[2020-11-20 22: 21: 28,232] ERROR - agent | Error while deploying flow: FileNotFoundError (2, 'No such file or directory')
Note: If you manually run the agent.py file in the folder with: dev $ python3 agent.py The agent works normally, could someone give me a light where I am failing?
👀 1
s

Scott Moreland

11/23/2020, 1:44 PM
Few things that might help. Python pathlib includes a glob command that you can use to simplify your filepath walk. I'd try printing the paths to stdout to make sure they: 1. point to the correct locations 2. are absolute filepaths
👍 1
y

Yuri Caruso

11/24/2020, 10:34 AM
Thanks, that's exactly what I did. I found the PATH that was running when calling the agent for the supervisor, so I found out that I can set a path on the supervisor itself. Setting there he has already found the prefect package.
[program:start_agent_prefect]
directory=/home/dev/projetos/data-automation/
environment=PATH="/home/dev/projetos/data-automation/venv/bin:/home/dev/.local/bin:/opt/sqlanywhere16/bin64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/dev/.local/bin:/snap/bin",LD_LIBRARY_PATH="/opt/sqlanywhere16/lib64"
command=prefect agent local start --import-path "/home/dev/projetos/data-automation/src"
user=dev
autostart=true
autorestart=true
stderr_logfile=/var/log/agent_prefect.err.log
stdout_logfile=/var/log/agent_prefect.out.log