https://prefect.io logo
Title
m

Mark Williams

03/12/2020, 9:30 PM
Three Questions: 1. Is there a way to do more with the visualizations that prefect core outputs? Like append a file name to a task? 2. Also, is there a way to split the files out dynamically? I adopted the visualization state handler from the examples, but when I run large amounts of loops through, it just makes a blank file. 3. How do I make the flow write to a log file? I have the flow result_handler set to LocalResultHandler with a directory specified, but it does not write anything to it.
c

Chris White

03/13/2020, 1:22 AM
Hi @Mark Williams - apologies for the delayed response! 1. Are you referring to the output of
flow.visualize()
? If so, there is not much you can do to customize this currently. 2. You’ll need to use some custom logic to split
flow.visualize
out into multiple files. Most of our users use
flow.visualize
as a sanity check, and rely on our UI’s visualization features during execution (e.g., https://docs.prefect.io/cloud/ui/flow-run.html#schematic) 3. Result handlers are the interface to configure how task outputs are handled. For writing logs to a file, you can attach a
FileHandler
to the prefect logger (https://docs.python.org/3/library/logging.handlers.html#filehandler) as follows:
import prefect
from logging import FileHandler

file_handler = FileHandler(filename="logs.txt")

root_prefect_logger = prefect.utilities.logging.get_logger()

root_prefect_logger.addHandler(file_handler)
m

Mark Williams

03/13/2020, 1:28 AM
@Chris White Bummer on item 1. Item 2. I really like having the visualized flow with the colors as a quick way of debugging. Item 3. Awesome! I will implement this tomorrow. Processing 2k+ files makes the output crazy in my IDE. Do you all have a support team that would help with implementation and such for companies looking to utilize Prefect?
c

Chris White

03/13/2020, 1:31 AM
For Item 1 (and even Item 2) if you can spec out what you’d like, feel free to open an enhancement request on github and we can look into exposing the functionality you’re looking for! And yes we absolutely can discuss support services with you; if you want to follow up on this feel free to send both myself and one of our SE’s an email at
chris @ <http://prefect.io|prefect.io>
and
kyle @ <http://prefect.io|prefect.io>