Hi all. I have a situation where my flow used mapped tasks. At times, though, there are no logs for some children tasks. Here is an example. The other interesrting things is that I can't reproduce it on my local machine -- it only seems to happen on prefect cloud
Marc Lipoff
01/04/2021, 9:19 PM
Marc Lipoff
01/04/2021, 9:19 PM
Marc Lipoff
01/04/2021, 9:19 PM
k
Kyle Moon-Wright
01/04/2021, 9:46 PM
Hey @Marc Lipoff,
Getting more insight on this however, this can occur using multi-threading / multi-processing in which the information passed via
prefect.context
(e.g., flow run ID, task run ID, map index, etc.) gets dropped when passed to a new thread or subprocess - task configurations like `timeout`s can cause this weird behavior to occur.
m
Marc Lipoff
01/04/2021, 10:15 PM
Great. Thanks @Kyle Moon-Wright let me know what you find.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.