Hello prefect friends I have an application that uses agents

Question

Hello prefect friends I have an application that uses agents and I m working on refactoring to leverage prefect workers The app is driven by docker docker compose with one service dedicated to agents Flows are deployed using the prefect deployment build command in a bash script kicked off when the docker compose is brought up I ve updated all deployment commands to prefect no prompt deploy and changed the docker compose entry point from prefect agent start q lt queue name gt to prefect worker start p lt pool name gt However when I attempt to run a flow I get an error message of file not found Am I missing something obvious in my set up I can share more info as needed Thanks in advance

Nate · Answer

hi < Joe B> do you have the trace im wondering which file it cant find you ll need to have your `prefect yaml` at the root of your repo

John Kang · Answer

Does it have to do with where you are storing your code

John Kang · Answer

Is it on Github GCS AWS etc

John Kang · Answer

Maybe it is not uploading correctly

Joe B · Answer

Hi < Nate> < John Kang> thanks for the responses I reverted my changes and am working on getting that stack trace back The prefect yaml was good info I don t think I was properly generating one My code is currently local for the initial refactor

Joe B · Answer

Here is the trace ```Worker ProcessWorker bc574b38 164d 423d 9fa9 3c78b6965e51 submitting flow run a8df70ee 63e7 4119 b9e7 0875ab4b4bb9 10 37 36 AM prefect flow runs worker Opening process 10 37 36 AM prefect flow runs worker Process running command usr local bin python m prefect engine in tmp tmpspe8bqgqprefect 10 37 36 AM prefect flow runs worker Completed submission of flow run a8df70ee 63e7 4119 b9e7 0875ab4b4bb9 10 37 36 AM prefect flow runs worker Running 1 deployment pull steps 10 37 37 AM prefect flow runs Changing working directory to opt prefect 10 37 37 AM prefect flow runs Importing flow code from flows ingestion py ingestion 10 37 37 AM prefect flow runs Flow could not be retrieved from deployment Traceback most recent call last File lt frozen importlib bootstrap external gt line 936 in exec module File lt frozen importlib bootstrap external gt line 1073 in get code File lt frozen importlib bootstrap external gt line 1130 in get data FileNotFoundError Errno 2 No such file or directory opt prefect flows ingestion py The above exception was the direct cause of the following exception Traceback most recent call last File usr local lib python3 11 site packages prefect engine py line 395 in retrieve flow then begin flow run flow = await load flow from flow run flow run client=client ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect client utilities py line 51 in with injected client return await fn args kwargs ^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect deployments deployments py line 222 in load flow from flow run flow = await run sync in worker thread load flow from entrypoint str import path ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities asyncutils py line 91 in run sync in worker thread return await anyio to thread run sync ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages anyio to thread py line 33 in run sync return await get asynclib run sync in worker thread ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages anyio backends asyncio py line 877 in run sync in worker thread return await future ^^^^^^^^^^^^ File usr local lib python3 11 site packages anyio backends asyncio py line 807 in run result = context run func args ^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect flows py line 975 in load flow from entrypoint flow = import object entrypoint ^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities importtools py line 201 in import object module = load script as module script path ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File usr local lib python3 11 site packages prefect utilities importtools py line 164 in load script as module raise ScriptError user exc=exc path=path from exc prefect exceptions ScriptError Script at flows ingestion py encountered an exception FileNotFoundError 2 No such file or directory 10 37 37 AM prefect flow runs Process 27 exited cleanly ```

Joe B · Answer

I m going to attempt to run `prefect init` to get a `prefect yaml` file and see what may or may not change

Joe B · Answer

Still erroring out I think migrating from agents to workers will be a heavier lift than I initially expected My bash script is managing the deployment build and I think I need to hand some of that responsibility over to the yaml file

John Kang · Answer

When I converted over from agents to workers it was a bit of a lift too

John Kang · Answer

how are you setting your remote code

John Kang · Answer

I set it up like this in the yaml ``` pull section allows you to provide instructions for cloning this project in remote locations pull prefect deployments steps git clone repository <https github com johnkangw site analytics git> TODO Update branch to master or main branch kubernetes testing access token prefect blocks secret deployment kubernetes worker weather process comp files repo token ```

John Kang · Answer

It was key for me to setup the access token otherwise I couldn t pull the code from github

John Kang · Answer

not sure how you are pulling your code

Joe B · Answer

Thanks John this is good info I m attempting to run things locally right now but will be pulling from github when things start shaping up

Nate · Answer

gt My bash script is managing the deployment build and I think I need to hand some of that responsibility over to the yaml file yes in the `deployments` section of the `prefect yaml` you can template and or hardcode in the stuff you would have passed into `prefect deployment build` more or less the important difference s being that you no longer need to specify infra storage block no more storage block required for deployment gt instead use a `pull` step to tell the worker where to get the code no more infra block required for deployment gt instead the work pool assigned to your deployment tells the worker how to execute the flow run