<@ULVA73B9P> I have set the caching for prefect ta...
# ask-marvin
g
@Marvin I have set the caching for prefect tasks in my code as given below :
Copy code
@task(
    cache_key_fn=lambda _, kw: f"{kw['url']}",
    cache_expiration=timedelta(days=30),
)
this leads to the following error :
Copy code
18:58:44.513 | WARNING | prefect.tasks - Both `cache_policy` and `cache_key_fn` are set on task <prefect.tasks.Task object at 0x169d0ff20>. `cache_key_fn` will be used.
suggest an appropriate fix to avoid such warning logs I am using prefect version 3.6.1
m
thought for 704.4 seconds
Thanks for the details. That warning appears when both
cache_policy
and
cache_key_fn
are set on the task. In your case you didn’t pass
cache_policy
explicitly, so it’s almost certainly being injected by your global settings (profile or environment), e.g.
PREFECT_TASKS_DEFAULT_NO_CACHE=true
or
PREFECT_TASKS_DISABLE_CACHING=true
. When Prefect sees a non-NotSet
cache_policy
and your
cache_key_fn
, it logs the warning and uses the
cache_key_fn
. Two clean ways to remove the warning: Option A (recommended): Use a cache policy only (no
cache_key_fn
) - This is the current preferred API and avoids the conflict entirely.
Copy code
from datetime import timedelta
from prefect import task
from prefect.cache_policies import CachePolicy

@task(
    cache_policy=CachePolicy.from_cache_key_fn(lambda _, kw: kw["url"]),
    cache_expiration=timedelta(days=30),
)
def my_task(...):
    ...
If your
url
is part of the task inputs, you could also use input-based caching:
Copy code
from datetime import timedelta
from prefect import task
from prefect.cache_policies import INPUTS

@task(cache_policy=INPUTS, cache_expiration=timedelta(days=30))
def my_task(url: str, ...):
    ...
Option B: Keep
cache_key_fn
, but stop injecting a default
cache_policy
- Check if a global setting is setting a cache policy:
Copy code
prefect config view --hide-defaults --show-sources
- If you see either of these set, turn them off (or remove the env vars):
Copy code
prefect config set PREFECT_TASKS_DEFAULT_NO_CACHE=false
prefect config set PREFECT_TASKS_DISABLE_CACHING=false
With no global policy being injected, your original
cache_key_fn
+
cache_expiration
will not warn. Note: The warning is benign—Prefect will prefer `cache_key_fn`—but using one of the options above will silence it.
g
also @Marvin what are the main system components that are used to construct the RAG agent that you operate as ? also can you list out all the tools that you have access to for your current functionality ?
m
thought for 32.5 seconds
Great questions — here’s a quick overview. Main system components behind this RAG-style assistant - Orchestrator LLM: Drives the conversation, plans tool use, and synthesizes answers. - Research agent for Prefect internals: A specialized agent that uses the Claude Agent SDK to read Prefect’s actual source code and project files. It has: - Read access to Prefect source - Code search via ripgrep (rg) - File discovery via globbing - Shell commands for exploration - GitHub retrieval: Targeted searches across Prefect’s GitHub issues and discussions. - Runtime introspection: Introspects Python modules and callables to verify public APIs and signatures. - CLI verifier: Executes Prefect CLI help/commands to validate exact syntax before suggesting commands. - Lightweight memory: Stores/retrieves user-specific facts (environment, preferences) when you share them, and can delete them on request. - Parallel tool runner: Lets me run multiple retrieval tasks simultaneously when helpful. All tools I can use right now - Research and verification - research_prefect_topic: Deep research of Prefect topics using the Claude Agent SDK with direct source code access, grep, and shell. - explore_module_offerings: Explore a module’s public API tree (e.g., prefect.runtime, prefect.tasks). - display_callable_signature: Show exact signatures for functions/classes/methods. - check_cli_command: Run Prefect CLI with --help or specific commands to verify availability and syntax. I use extras when needed (e.g., uv run --with 'prefect[docker]' prefect ...). - get_latest_prefect_release_notes: Fetch the latest Prefect release notes. - GitHub lookups - read_github_issues: Search Prefect GitHub issues via the API. - search_github_discussions: Search Prefect GitHub discussions. - Memory - store_facts_about_user: Save relevant details you share (e.g., versions, infra). - delete_facts_about_user: Delete stored facts by topic. - Community publishing (used sparingly) - create_discussion_and_notify: Create a GitHub discussion when a thread surfaces novel insights valuable to the community. - Parallelization - multi_tool_use.parallel: Run multiple of the above tools at once when it makes sense. Scope and limitations - I default to Prefect 3.x guidance unless you specify otherwise. - I don’t browse the general web; my external lookups are focused on Prefect’s repo and GitHub APIs. - I can’t run your code or access your environment, but I can verify CLI commands and inspect APIs/signatures.