Deceivious
03/06/2023, 9:17 AMcache_key_fn=task_input_hash,
cache_expiration=datetime.timedelta(hours=2),
persist_result=True,
result_storage=LocalFileSystem(), # Change to remote in future
result_serializer=JSONSerializer()
For every task call , I want to log the cache_key
, if the task returned the cached result OR if the result were cached. Is this supported by prefect by default? / What are the best practices for this type of logging to be implemented?Filip Panovski
03/06/2023, 9:24 AMDeceivious
03/06/2023, 9:26 AMtask_input_hash
to log the cache key
but I have no way of knowing if the result was returned from the cache or not.Filip Panovski
03/06/2023, 9:29 AMDEBUG
and check if the caching layer logs anything additionally. If so, you could set the log level of only the cache class to DEBUG
Deceivious
03/06/2023, 9:35 AMChristopher Boyd
03/06/2023, 2:18 PMprint_logs=True
to your task / flow decorators to add print statements to your log output.
Additionally, 2.7.11
allowed the use of custom loggers -
https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-2711Deceivious
03/06/2023, 2:20 PMChristopher Boyd
03/06/2023, 2:25 PMDeceivious
03/06/2023, 2:46 PMChristopher Boyd
03/06/2023, 2:50 PMDeceivious
03/06/2023, 3:01 PMstate
is stored on the database [previously orion
] , The result itself is stored in memory OR defined blocks depending on how that task decorators are configured.
But what I want to know is
if the flow context has any information on if the called task actually ran or took the result out of the storage [cache].
For example: in API, the state_name field changes based on the state and if the results were cached.
[left pic : Cached , right pic : completed] .
I want to get this in the flow code so i can send a warning if we received cached data.Christopher Boyd
03/06/2023, 3:18 PMDeceivious
03/06/2023, 3:19 PM16:19:31.273 | INFO | Task run '_send_request_with_retry-1' - Finished in state Cached(type=COMPLETED)
Christopher Boyd
03/06/2023, 3:20 PMDeceivious
03/06/2023, 3:26 PMreturn_state
when calling the task, it does seem to have the state name saying it is Cached
. I am not sure if the intentended usage of this name is for this purpose though.