Are there any resources out there that explain how the prefect DAG-less approach works and the ideas behind it?
02/14/2023, 3:02 AM
@justabill has a piece in the works that might be what you're looking for.
soon horse 1
02/14/2023, 3:40 AM
I would be really interested in seeing it. I demoed some of my prefect workflows to the larger department and got some very pointed questions about how prefect / orion handles the state transitions, and what type of control structures are actually supported. I am aware that prefect avoids a DAG (which is what the other folks were really asking about), but what it actually is rely on I have no clue.
02/14/2023, 4:20 PM
Ah, that’s more technical than Bill’s piece was aiming, I believe.
Re States: this table explains them all, but not the mechanisms behind tracking. The API docs show the source code if you expand them (or you could jump into the repo). I don’t know of a diagram/post on the topic.
On control structures, most anything you can do in Python is supported, I believe. Looping and branching are the two that come to mind. So if and if else, while, for, loops are all fine.
Not exactly what you’re asking, but returning a generator from a task fails at the moment due to an upstream dependency issue with AnyIO that should be resolved when they merge a PR. See discussion here.
02/14/2023, 6:49 PM
@Jeff Hale linked the best resources for the answers to the ore specific/pointed questions. The blog post we'll be publishing later this week is higher level and more philosophical, but it might be a good complement. I'll circle back here and link to it once its up.
02/15/2023, 1:56 AM
This is all exactly what I was after -- thanks for the prompt and detailed feedback 🙂