Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Prefect & Controlflow look like an awesome combo and I love the clean API of CF and also the possibilities it opens up.

I have a use case where I read in file paths, break them down into parts ie version, department etc and then want to map them onto tasks in a production tracking system.

I want to learn from what users did the last 100 times and suggest them the right tasks to map too with AI.

This means I have a dataset of file parts and what was the wrong and the right choice in the past.

So I could train a custom AI but I'm also wondering if CF might be a solution?

Can I create a large context and provide it to an agent as training /knowledge?

Can I vectorize the content to reduce the context as it has a lot of repetition?

It sounds like memory might be the way to go?

If yes how would you structure the dataset to be most efficient? Ie one memory vs multiple?

And then a list of lines with the parts, the mapping and if it was correct or incorrect in text format for the AI to learn from? 

And if it is better to train a custom AI could I incorporate my custom AI in a workflow with tasks with open AI with CF?