Hey all, has anyone looked into using a progress bar like

tqdm

inside a task? I know there would be issues around stdout and how the prefect logger works but maybe someone has thought about this.

Hi @Alex Papanicolaou - I haven't tried this personally but since logging events aren't updatable I would imagine something like this wouldn't work with the Prefect Logger

hey alex, we do this but it's not super easy this is what we use to do tqdm->logger

class TqdmToLogger(StringIO):
    """Mock file handle to allow tqdm progress bar to write to logger.

    Adapted from <https://github.com/tqdm/tqdm/issues/313#issuecomment-267959111>.
    """

    def __init__(self, logger=None, level=<http://logging.INFO|logging.INFO>):
        self.buf = ""
        self.logger = logging.getLogger() if logger is None else logger
        self.level = level
        super().__init__()

    def write(self, buf):
        self.buf = buf.strip("\r\n\t ")

    def flush(self):
        self.logger.log(self.level, self.buf)


class tqdm(_tqdm):
    """Wrapper around `tqdm.auto` that plays nicely with `fluentd` and Prefect logging."""

    def __init__(self, iterable=None, *args, file=None, **kwargs):
        if file is None and "notebook" not in _tqdm.__module__:
            file = TqdmToLogger()
        super().__init__(iterable, *args, file=file, **kwargs)

we also have some magic that tries to make the default logger write to both prefect and stdout (not stderr in our case because we want the logs to show up in Stackdriver too) but I'm in the process of reworking that right now bc it doesn't always work...