https://prefect.io
Join Slack
Does controlflow leverage openai's structured outputs when it maps to pydantic result types?
# marvin-ai
j
Does controlflow leverage openai's structured outputs when it maps to pydantic result types?
a
At the moment it’s functioning calling under the hood. Structured output API is great but the initial registration step is a little too slow for my liking ATM
j
Whats the latency you're seeing there?
Also, is it function calling with strict mode?
a
For pydantic models with >15 fields or ive seen a time to first token at the registration step be near 8 seconds :(
But happy to make it an option for the bold lol
Are you getting parsing errors or somethng?
j
nope just curious!!
Im mostly wanting to know the structured output guarantees
a
Sounds good. We should probably write that down somewhere
j
I haven't tested since it came out but when it was introduced I found structured mode not as flexible as function calling in terms of reading the full json schema (stuff like field descriptions)
IIRC to use structured outputs you just need a 
strict=True
flag on the function definition so we could definitely find a way to let you indicate that on CF tools
not sure if its actually worth it
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by