Hello, has anyone seen this warning before? Basica...
# ask-community
m
Hello, has anyone seen this warning before? Basically one of my tasks is taking long time to even run. There is nothing special about this task and I've used it in other scripts /projects and have not had this issue before. Could it be because I recently upgraded to
2.18.0
?
Copy code
Task parameter introspection took 65.871 seconds , exceeding `PREFECT_TASK_INTROSPECTION_WARN_THRESHOLD` of 10.0. Try wrapping large task parameters with `prefect.utilities.annotations.quote` for increased performance, e.g. `my_task(quote(param))`. To disable this message set `PREFECT_TASK_INTROSPECTION_WARN_THRESHOLD=0`
If it matters, this task just takes in a list returned from another task, creates a dataframe out of it and does something with it.
m
If you're passing large dataframes directly between tasks, this can affect performance. Can you try using external storage for your dataframes and referencing that storage from your functions? https://docs.prefect.io/latest/guides/big-data/?h=big+d#save-data-to-disk-within-a-flow