t

    Tomer Cagan

    6 months ago
    Orion question (applies to 1.0 as well I guess): In the docs here, it is stated that tasks can only be called from a flow and not other tasks: "All tasks must be called from within a flow. Tasks may not be called from other tasks." So this means I can only do dynamic task spawning from within the flow function (Flow block in 1.0)? Also, I was working yesterday through the Orion's getting started guide and had an error and found a few typos - where should I report this?
    Anna Geller

    Anna Geller

    6 months ago
    That's correct, you can only call tasks in a flow, but with Orion, you can then also call flows in other flows (subflows), so there's a huge amount of flexibility here šŸ™‚ Bug reporting always occurs in GitHub issues: https://github.com/PrefectHQ/prefect/issues/new/choose There is a special category for Orion:
    t

    Tomer Cagan

    6 months ago
    Is it legal for a flow to call itself? (recursively)
    Anna Geller

    Anna Geller

    6 months ago
    Even if it would be possible, why would you do that? Can you start by describing your use case/the problem that you try to solve?
    t

    Tomer Cagan

    6 months ago
    As I stated before (maybe in intro), we have an algorithm that tries to do some compute-heavy calculation, and it case it fails, split the problem into smaller parts and call the same logic. I think maybe this is too low level for what prefect / orion is aiming to solve but the abstractions provided by prefect seem promising so trying to figure out if that would somehow be possible...
    Anna Geller

    Anna Geller

    6 months ago
    That's definitely possible šŸ™‚ However, you can solve that without recursion, but instead using e.g. subflows. To give you an idea of what's possible already in Prefect 1.0, here is an example parent flow triggering different child flows (subflows) based on upstream task/child-flow's state. So if your flow succeeds, you could spin up one subflow and if it fails, trigger a different subflow. You can do all that already in 1.0. And to reduce the code duplication between those two child flows, instead of relying on recursion, you can build a reusable module that can be called in both subflows.
    actually, Dylan suggested another useful option you could look into: instead of recursion and running various different subflows from a parent flow, you could call the same flow multiple times with various parameter values using the
    create_flow_run
    task - basically the call would call itself with various parameter values depending on some condition (e.g. upstream task's state)
    Michael Adkins

    Michael Adkins

    6 months ago
    Orion will support recursion although I do not know if Iā€™d recommend it.
    I think right now it will not work since the task runner would be reused.