https://prefect.io logo
Title
k

Kyle McChesney

10/18/2022, 2:29 PM
Rather interesting task failure. I have the following resource manager (some functions removed to keep the example simple):
@resource_manager
class LocalFileManager:

    ...

    def setup(self) -> 'LocalFileManager':
        self.temp_dir = Path(mkdtemp())
        self.logger = prefect.context['logger']
        <http://self.logger.info|self.logger.info>('Set up LocalFileManager @ %s', self.temp_dir)
        self._lock = RLock()
        self._cache = {}
        return self

    def cleanup(self, _) -> None:
        with self._lock:
            <http://self.logger.info|self.logger.info>('Cleaning up: %s', self.temp_dir)
            shutil.rmtree(self.temp_dir)
I had a flow fail, the setup task is the root failure. Here are the logs
17 October 2022,06:09:30 	prefect.CloudTaskRunner	INFO	Task 'LocalFileManager.setup': Starting task run...
17 October 2022,06:09:30 	prefect.LocalFileManager.setup	INFO	Set up LocalFileManager @ /tmp/tmp1b3_t4xs
17 October 2022,06:09:31 	prefect.CloudTaskRunner	INFO	Task 'LocalFileManager.setup': Finished task run for task with final state: 'Success'
17 October 2022,08:55:01 	prefect.CloudTaskRunner	INFO	Task 'LocalFileManager.setup': Starting task run...
17 October 2022,08:55:01 	prefect.CloudTaskRunner	ERROR	Task 'LocalFileManager.setup': Exception encountered during task execution!
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 884, in get_task_run_state
    logger=self.logger,
  File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 468, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/usr/local/lib/python3.7/site-packages/prefect/tasks/core/resource_manager.py", line 25, in run
    return mgr.setup()
AttributeError: 'NoneType' object has no attribute 'setup'
17 October 2022,08:55:01 	prefect.CloudTaskRunner	INFO	Task 'LocalFileManager.setup': Finished task run for task with final state: 'Failed'
It seems like the task ran twice and fails for the second run. This exact flow has run multiple times without issue, and this is a common resource manager used in multiple other flows.
1️⃣ 1