https://prefect.io
Join Slack
Hello, i was wondering if there's any way we can look at the output of each step using prefect or so...
# ask-community
n
Hello, i was wondering if there's any way we can look at the output of each step using prefect or something similar to the python fn-graph (Python modelling pipeline library) ?
k
Hey @Nishtha Varshney, you can log the task output before the return. Let me check that library though so I really know what you are talking about. I have not seen it yet
You mean you want the graph? You can try doing 
flow.visualize
?
n
@Kevin Kho i know about flow.visualise i wanted more info say for example if u train a ml model u can see the summary using model.summary or a table of all the details is it possible to get smething similar using prefect?
k
Gotcha. Have you seen Artifacts? You can create a Markdown artifact with that info, which will then appear in the Prefect UI attached to that flow.
👍 1
n
Yes i tried creating a markdown of the model.summary it said its 'None' even in the logs i get the same thing
k
Could you show me your flow?
n
Copy code
MARKDOWN2="""
Summary : {Summary}
"""
@task
def model(x_train,y_train,x_test,y_test):
        num_classes = 10
        input_shape = (28, 28, 1)
        model = tensorflow.keras.Sequential(
            [
                tensorflow.keras.Input(shape=input_shape),
                tensorflow.keras.layers.Conv2D(32, kernel_size=(3, 3), activation="relu"),
                tensorflow.keras.layers.MaxPooling2D(pool_size=(2, 2)),
                tensorflow.keras.layers.Conv2D(64, kernel_size=(3, 3), activation="relu"),
                tensorflow.keras.layers.MaxPooling2D(pool_size=(2, 2)),
                tensorflow.keras.layers.Flatten(),
                tensorflow.keras.layers.Dropout(0.5),
                tensorflow.keras.layers.Dense(num_classes, activation="softmax"),
            ]
        )
        logger = prefect.context.get("logger")
        <http://logger.info|logger.info>(model.summary())
        prefect.artifacts.create_markdown(MARKDOWN2.format(Summary=model.summary()))
This is a part of the flow
k
Been a while since I touched tensorflow. What are you expecting to see with 
model.summary()
? Is it None because training didn’t happen yet? Or is summary supposed to print the layers?
n
So model.summary() prints the Name and type of all layers in the model, output shape for each layer, number of weight parameters etc. After training the model the testing works too, so model is trained correctly i wanted to knw if prefect has any option to print all these details too ?
k
It should. I don’t know why logging the summary would be None.
n
It works, by converting the dataframe to string
k
model.summary()
returns a dataframe?
n
No sorry it returns the object which can be converted into a string
Copy code
stringlist = []
model.summary(print_fn=lambda x: stringlist.append(x))
short_model_summary = "\n".join(stringlist)
I used this code to convert it into a string
k
Gotcha! Thanks!
✅ 1
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by