Hey y'all, I have a task where I am running a command in a docker container. This task ends up taking a while and I have a decent amount of logging that I'd like to stream back to the cloud UI. For some reason, the logs only come through after the task is complete. Is there a way to stream this logging back to prefect cloud?

@task()
def docker_command(docker_image, environment_variables, command, logger):
    response = (
        docker.from_env()
        .containers.run(
            image=docker_image,
            command=f"python {command}",
            environment=environment_variables,
        )
        .decode("utf-8")
    )
    <http://logger.info|logger.info>(response)

Also, I'm using the docker-py python library for this, rather than a block which might also be a solution?

There’s a prefect-docker collection: https://prefecthq.github.io/prefect-docker/#pull-image-and-create-start-log-stop-and-remove-docker-container