While registering the flow it is not storing the configs fil

Question

Anna Geller · Answer

Can you move code blocks to the thread

Raviraja Ganta · Answer

Getting this error ``` message Task configure paths task Starting task run levelname INFO filename task runner py module task runner lineno 241 funcName run flow name Clusters Creation Flow flow run id 9508bc8f 454e 4b62 937b 88e30df3f03b task name configure paths task task slug configure paths task 1 task run id 8cca14d2 f97e 4594 a617 aa567e363fbf timestamp 2022 05 05T10 50 21 913671+00 00 message Task configure paths task Exception encountered during task execution exc info Traceback most recent call last n File usr local lib python3 8 site packages prefect engine task runner py line 880 in get task run state n value = prefect utilities executors run task with timeout n File usr local lib python3 8 site packages prefect utilities executors py line 468 in run task with timeout n return task run args kwargs type ignore n File w python monorepo python monorepo taxonomy flows clusters py line 25 in configure paths task n File usr local lib python3 8 site packages hydra compose py line 33 in compose n cfg = gh hydra compose config n File usr local lib python3 8 site packages hydra internal hydra py line 559 in compose config n cfg = self config loader load configuration n File usr local lib python3 8 site packages hydra internal config loader impl py line 141 in load configuration n return self load configuration impl n File usr local lib python3 8 site packages hydra internal config loader impl py line 227 in load configuration impl n self ensure main config source available n File usr local lib python3 8 site packages hydra internal config loader impl py line 129 in ensure main config source available n self missing config error n File usr local lib python3 8 site packages hydra internal config loader impl py line 103 in missing config error n raise MissingConfigException nhydra errors MissingConfigException Primary config directory not found nCheck that the config directory w python monorepo python monorepo taxonomy config exists and readable levelname ERROR filename task runner py module task runner lineno 910 funcName get task run state flow name Clusters Creation Flow flow run id 9508bc8f 454e 4b62 937b 88e30df3f03b task name configure paths task task slug configure paths task 1 task run id 8cca14d2 f97e 4594 a617 aa567e363fbf timestamp 2022 05 05T10 50 22 295499+00 00 ```

Anna Geller · Answer

I thought you would come back because the solution you posted in the other thread doesn t solve the problem of packaging flow code it only sets credentials I can only give the same answer as <https prefect community slack com archives CL09KU1K7 p1651699738142469 thread ts=1651697518 396919 amp cid=CL09KU1K7|here> that you need to package this code including those config files into your Docker image you would need ```COPY configs opt prefect configs ```

Raviraja Ganta · Answer

< Anna Geller> what storage should i use here I am trying to print the flow file path when running in agent ``` task def print cwd logger = prefect context get logger os getcwd print os getcwd os path dirname file ``` It is printing os getcwd value as ` app` But somehow it is printing os path dirname as the path of the file in localsystem So though I have configs folder present in app configs it is not able to read it and is looking at lt localsystem path gt configs ``` message app levelname INFO filename clusters py module clusters lineno 44 funcName print cwd flow name Clusters Creation Flow flow run id 493988b6 a7f0 4b81 a0c5 9332d623c4ca task name print cwd task slug print cwd 1 task run id 98a63ed2 2153 44e0 a7c4 86b27eafb7ed timestamp 2022 05 05T16 53 28 056764+00 00 message Users raviraja Desktop python monorepo taxonomy flows levelname INFO filename clusters py module clusters lineno 47 funcName print cwd flow name Clusters Creation Flow flow run id 493988b6 a7f0 4b81 a0c5 9332d623c4ca task name print cwd task slug print cwd 1 task run id 98a63ed2 2153 44e0 a7c4 86b27eafb7ed timestamp 2022 05 05T16 53 28 057057+00 00 ```

Anna Geller · Answer

you could actually set this in your Dockerfile using the WORKDIR command The default working directory is opt prefect if you leverage base Prefect image What base image do you use app is used in Python base image usually

Raviraja Ganta · Answer

```FROM prefecthq prefect latest python3 8 RUN apt get update amp amp apt get install y git amp amp apt get y install gcc amp amp apt get install wget y COPY app WORKDIR app COPY requirements txt requirements txt RUN pip install r requirements txt no cache dir RUN pip install ```

Anna Geller · Answer

thanks In that case to add configs you may do that explicitly using ```COPY configs app configs ```

Raviraja Ganta · Answer

this will do that already `COPY app` I added a task to print the items of dir app ``` setup py config mlinfra tests lambdas notebooks flows iolib dataprocessing gpt dockerfile README md requirements txt taxonomy ``` somehow the flow is printing path as ` Users raviraja Desktop python monorepo taxonomy flows ` for this log statement `<http logger info|logger info> os path dirname file `

Anna Geller · Answer

probably an easier way of testing this would be to exec into the container and browse the file structure this way

Raviraja Ganta · Answer

the container does not contain flow code right it should be fetched from storage dynamically whenever a flow is scheduled and the container dies once the flow is done right

Anna Geller · Answer

it depends on how you build the image usually no your image doesn t need to contain your flow this is what Storage is for

Anna Geller · Answer

and yes when the flow run finishes the container finishes too