How to set `PREFECT LOGGING LOG ATTRIBUTES` I am trying to s

Question

How to set `PREFECT LOGGING LOG ATTRIBUTES` I am trying to send a unique id while creating a flow run using python client I want this uuid to be available in logs for tracking the request flow However so far I haven t had any success in correctly setting the extra log attributes This is how i start the docker agent ```prefect agent start docker e PREFECT LOGGING LOG ATTRIBUTES= uuid ``` I tried debugging and found that the log attributes are fetched as string <https github com PrefectHQ prefect blob 0 12 1 src prefect utilities logging py L182|here> instead of list

Chris White · Answer

Hi Sandeep can you describe what error you are seeing This worked for me ``` start an ipython session with env vars PREFECT LOGGING LOG ATTRIBUTES= uuid PREFECT LOGGING FORMAT= message s | uuid s ipython``` and then ```import prefect logger = prefect utilities logging get logger with prefect context uuid= 42 logger debug hi ``` I get ```hi | 42``` as I expect

Sandeep Aggarwal · Answer

Thanks for your reply < Chris White> Yes this works as expected when running locally However when using server with docker agent and dask cluster running on my local machine I see a strange behaviour I start agent with below command and also set prefect logging config in my worker environment variables everything works fine ```prefect agent start docker v f e PREFECT LOGGING LOG ATTRIBUTES= uuid e PREFECT LOGGING FORMAT= message s | uuid s network prefect logging default This is dask worker service in my compose file dask worker image daskdev dask command dask worker dask scheduler 8786 extra hosts host docker internal 172 17 0 1 environment PREFECT LOGGING LEVEL=DEBUG PREFECT LOGGING LOG ATTRIBUTES= uuid PREFECT LOGGING FORMAT= message s | uuid s Agent output Beginning Flow run for logging flow | None Starting flow run | 42 Flow logging flow Handling state change from Scheduled to Running | 42 Flow run SUCCESS all reference tasks succeeded | 42 Flow logging flow Handling state change from Running to Success | 42 Worker output distributed core INFO Starting established connection Task log with attrs Starting task run | 42 Task log with attrs Handling state change from Pending to Running | 42 Task log with attrs Calling task run method | 42 hi | 42 Task log with attrs Handling state change from Running to Success | 42 Task log with attrs finished task run for task with final state Success | 42``` But when logging config is not passed to agent The logging config is still set in worker environment though below exception is raised The flow still runs as expected

Sandeep Aggarwal · Answer

```prefect agent start docker v f network prefect logging default This is dask worker service in my compose file dask worker image daskdev dask command dask worker dask scheduler 8786 extra hosts host docker internal 172 17 0 1 environment PREFECT LOGGING LEVEL=DEBUG PREFECT LOGGING LOG ATTRIBUTES= uuid PREFECT LOGGING FORMAT= message s | uuid s Agent output 2020 06 29 02 53 18 INFO prefect CloudFlowRunner | Beginning Flow run for logging flow 2020 06 29 02 53 18 INFO prefect CloudFlowRunner | Starting flow run 2020 06 29 02 53 18 DEBUG prefect CloudFlowRunner | Flow logging flow Handling state change from Scheduled to Running 2020 06 29 02 53 18 INFO prefect CloudFlowRunner | Flow run SUCCESS all reference tasks succeeded 2020 06 29 02 53 18 DEBUG prefect CloudFlowRunner | Flow logging flow Handling state change from Running to Success Docker Worker logs File usr local lib python3 7 threading py line 926 in bootstrap inner self run File usr local lib python3 7 threading py line 870 in run self target self args self kwargs File usr local lib python3 7 site packages distributed threadpoolexecutor py line 55 in worker task run File usr local lib python3 7 site packages distributed concurrent futures thread py line 65 in run result = self fn self args self kwargs File usr local lib python3 7 site packages distributed worker py line 3419 in apply function result = function args kwargs File usr local lib python3 7 site packages prefect engine flow runner py line 696 in run task context=context File usr local lib python3 7 site packages prefect utilities executors py line 257 in wrapper return func args kwargs File usr local lib python3 7 site packages prefect engine cloud task runner py line 315 in run is mapped parent=is mapped parent File usr local lib python3 7 site packages prefect utilities executors py line 257 in wrapper return func args kwargs File usr local lib python3 7 site packages prefect engine task runner py line 291 in run state = self get task run state state inputs=task inputs File usr local lib python3 7 site packages prefect utilities executors py line 66 in inner return runner method self args kwargs File usr local lib python3 7 site packages prefect engine runner py line 48 in inner new state = method self state args kwargs File usr local lib python3 7 site packages prefect engine task runner py line 822 in get task run state self task run timeout=self task timeout raw inputs File usr local lib python3 7 site packages prefect utilities executors py line 186 in timeout handler return fn args kwargs File home sandeep workspace playground prefect logging logging flow py line 10 in log with attrs Message hi Arguments Logging error Traceback most recent call last File usr local lib python3 7 logging init py line 1025 in emit msg = self format record File usr local lib python3 7 logging init py line 869 in format return fmt format record File usr local lib python3 7 logging init py line 611 in format s = self formatMessage record File usr local lib python3 7 logging init py line 580 in formatMessage return self style format record File usr local lib python3 7 logging init py line 422 in format return self fmt record dict KeyError uuid ```

Sandeep Aggarwal · Answer

These are logs from UI s flow run > logs tab which doesn t show any errors

Chris White · Answer

Interesting OK yea occasionally logging configuration is not properly passed on to dask workers so this sounds like a possible bug I ll open an issue for you and we can move this conversation there

Chris White · Answer

< Marvin> open Adding logging attributes raises error when running on dask

Marvin · Answer

Sandeep Aggarwal · Answer

Great Thanks < Chris White> I will follow the progress there