is there any open source alternative to prefect cloud?

Hi! Any specific reason you are in need of an open source alternative?

I want to use the dynamic visualization feature of visualization, my needs will be larger than a free tier requirement and I can't afford the pricing, is there some free open source alternative to prefect cloud where I can use the prefect core?

Gotcha, for the dynamic visualization point I think someone else has mentioned dask UI (https://distributed.dask.org/en/1.12.2/web.html#example-computation) if you are using the dask executor, but dask doesn’t know how the flows are organized, so its more like visualizing overall task throughput than how the flows are going. I don’t personally know of any published open source alternatives to the dynamic visualization of Prefect flows like in Cloud/Cloud UI (maybe someone else will jump in with one or one in progress). I know at least one other person in this channel was interested in something like flow.visualize() to depict states of flows while they are running, which sounds a bit like the usage you’re interested in. Internally we are also considering a terminal-based UI like k9s (https://github.com/derailed/k9s#screenshots) in core to watch and report on flows that are happening in-process. Let me know if either of those are the kind of thing you are looking for so we can prioritize it in core better.

can prefect core be used with dagster together?

Great, I will keep you posted about progress on the core terminal UI or enriching flow.visualize() projects. Right now they exist as whiteboard ideas for us, but once there are github issues I will tag you. As for prefect + dagster, I haven’t tried, but I think dagster’s UI/scheduling features depend on your pipelines being described with their API (as in decorated as

@solid

or

@pipeline

as opposed to Prefect’s

@task

and

with Flow

, etc). Is it that you want to use Prefect’s engine API or task library (or something), but want Dagster’s UI? To be honest though I haven’t dug into dagster too much so I can give it a whirl next week and be able to comment more intelligently afterwards 🙂

Yes use Prefect engine API with Dagster UI, also is it possible to have a workflow withing a workflow, and have some parallel workflows, I couldn't find examples for the same.