Does anybody have an example of running a flow from a serialized DAG representation? I want to be able to create a flow (in json) from a predefined set of tasks. I also want to support passing data from one task to another.
n
Nate
11/04/2024, 2:41 PM
hi @Evan Curtin - you may already know this, but just as a disclaimer, prefect (post prefect 1.x anyways) discovers the graph, so in most cases you shouldn't need a static dag representation, as python control flow can typically represent the same work
but there's different ways to do this, here's one. but IME these rely on at least
• defining some task registry (mapping of tasks you can run)
• making up some DSL to define that downstream tasks should get inputs from an upstream result that you don't know yet
e
Evan Curtin
11/04/2024, 2:42 PM
ty for the link I'll take alook. I don't want the end users to be writing code so I'd prefer a DSL
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.