Deceivious03/06/2023, 9:17 AM
For every task call , I want to log the
cache_key_fn=task_input_hash, cache_expiration=datetime.timedelta(hours=2), persist_result=True, result_storage=LocalFileSystem(), # Change to remote in future result_serializer=JSONSerializer()
, if the task returned the cached result OR if the result were cached. Is this supported by prefect by default? / What are the best practices for this type of logging to be implemented?
Filip Panovski03/06/2023, 9:24 AM
Deceivious03/06/2023, 9:26 AM
to log the
but I have no way of knowing if the result was returned from the cache or not.
Filip Panovski03/06/2023, 9:29 AM
and check if the caching layer logs anything additionally. If so, you could set the log level of only the cache class to
Deceivious03/06/2023, 9:35 AM
Christopher Boyd03/06/2023, 2:18 PM
to your task / flow decorators to add print statements to your log output. Additionally,
allowed the use of custom loggers - https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-2711
Deceivious03/06/2023, 2:20 PM
Christopher Boyd03/06/2023, 2:25 PM
Deceivious03/06/2023, 2:46 PM
Christopher Boyd03/06/2023, 2:50 PM
Deceivious03/06/2023, 3:01 PM
is stored on the database [previously
] , The result itself is stored in memory OR defined blocks depending on how that task decorators are configured. But what I want to know is
For example: in API, the state_name field changes based on the state and if the results were cached. [left pic : Cached , right pic : completed] . I want to get this in the flow code so i can send a warning if we received cached data.
if the flow context has any information on if the called task actually ran or took the result out of the storage [cache].
Christopher Boyd03/06/2023, 3:18 PM
Deceivious03/06/2023, 3:19 PM
16:19:31.273 | INFO | Task run '_send_request_with_retry-1' - Finished in state Cached(type=COMPLETED)
Christopher Boyd03/06/2023, 3:20 PM
Deceivious03/06/2023, 3:26 PM
when calling the task, it does seem to have the state name saying it is
. I am not sure if the intentended usage of this name is for this purpose though.