Hi all, I am trying to understand performance issues I get when I run great expectations task
from prefect_great_expectations import run_checkpoint_validation
My data is small and medium csv files.
After debugging code step by step, I can see that GE prints results very quickly, in a few seconds, but then my code is stuck for a couple of minutes here `\Lib\site-packages\prefect\_internal\concurrency\waiters.py`:
Copy code
def _handle_waiting_callbacks(self):
logger.debug("Waiter %r watching for callbacks", self)
while True:
callback: Call = self._queue.get()
if callback is None:
break
All other non-GE tasks are fine here.
I really don't know where to go from here and appreciate any help
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.