Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi, is there a way to create a workflow by reading all tasks and dependencies from a json/xml ?

<@UKTUC906M> Can you please help me with this. Thanks.

Hi <@UM91DM491>, sorry I missed this before. We don’t ship a way to do this right now, but since Prefect is pure-Python, you could write a parser yourself that loaded the JSON and programmatically generated the flow.

Hey <@UKTUC906M>, Thanks for answering. I already did it using a parser and a program. I was just curious if there was already some way in prefect library to do it.

We’re going to lock down the Python API and then start working on bindings. We already use JSON as a serialization layer, so there’s some groundwork — but unfortunately we aren’t shipping any builtin parser yet, sorry.

<@UKTUC906M> Can we change the tasks or dependencies in a workflow when it's running ?

And how to use ifelse/switch in imperative flow ?

<@UM91DM491> your general workflow must be defined before it starts. However there are tools like `ifelse` and `map` that can help you take different paths through the workflow or generate dynamic tasks.

Great question about using `ifelse` in an imperative flow — we should document this in a FAQ. Because functions like `ifelse` generate intermediate tasks, I’d actually recommend switching to the functional API just for that one call. Something like:
```
# ... imperative flow building
flow.add_task(condition)
# switch to functional for ifelse
with flow:
    result = ifelse(condition, &lt;true&gt;, &lt;false&gt;)
# back to imperative
flow.add_task(...)
```
you *could* introspect ifelse, rebuild it yourself, and capture all the intermediate tasks, but this will accomplish the same thing. You can switch between functional and imperative APIs at any time.

However this makes me think we could just add an optional `flow` argument to `ifelse` so it could be used in the imperative API without modification. (cc <@UKNSNMUE6> for opinion?)

Yea I’m on board with that- I think maintaining compatibility with both APIs is important. Let’s open an issue for the ifelse task!

Hey guys, when I try to add a new task to a running workflow, its giving me KeyError. I am able to do `task.run()`. But when using `add_task`, it's giving me error. I even tried `task.set_upstream`. It's also giving the same error.

I am trying to design a dynamic workflow in which I'll be able to add any task to a running workflow at any time.

Hi <@UM91DM491> - Prefect does not support fully dynamic workflows like you are describing (yet)

Okay. Thanks. When will the support for dynamic workflows be available in prefect ?

Honestly can’t say - I wouldn’t expect it anytime soon for _arbitrary_ dynamic flows like you are describing; could you explain what use case you have that requires this level of customization?

We thought that we had that requirement for dynamic workflow. But now we think we can manage with the existing workflow that we have. Thanks for your help.