https://prefect.io logo
Title
a

Austin Weisgrau

02/24/2023, 10:11 PM
prefect.get_run_logger()
raises
MissingContextError
when called in a method called by
concurrent.futures.ThreadPoolExecutor
. Is this a bug or is it a prefect anti-pattern to use ThreadPoolExecutor within a prefect task?
from concurrent.futures import ThreadPoolExecutor
From prefect import task, flow, get_run_logger

def concurrent_subtask() -> None:
    # This raises MissingContextError
    get_run_logger()

@task
def basic_task():
    get_run_logger().info("This works.")
    with ThreadPoolExecutor(max_workers=2) as executor:
        futures = []
        for _ in range(2):
            future = executor.submit(concurrent_subtask)
            futures.append(future)

        for future in futures:
            future.result()

@flow(name="Hello World")
def helloworld() -> None:
    basic_task()
Discussion happening here