I'm new to Prefect, and have been experimenting with running flows on my local machine with a local server as well. So far things have been very intuitive! However I have another nodejs app running locally, and want to try to have that app call the local Prefect server to start a parameterized flow. I can start flows through the UI at localhost:8080 and I've been going through the documentation but things just haven't clicked and I'm not sure how to do this. (Additional disclaimer - I have virtually no experience with GraphQL which probably isn't helping). Thanks for the help and this amazing product!

Hey all, has anybody successfully started a dask cluster + prefect on top of an existing spark cluster (databricks)? I have a databricks cluster but want to use distributed prefect to schedule

I want to ask the Dask Experts whether they can help identify a scheduler bottleneck I have experienced. I have a flow that schedules 22 parallel mapped jobs (each with 65 items) allowing for 1430 parallel jobs. When I schedule the job on an k8s cluster with 375 nodes (each with a worker that runs 4 threads), my scheduler throws messages out like the below. The CPU utilization of the scheduler also goes to 100%. These jobs ultimately die when all the workers get disconnected.

distributed.utils_perf - WARNING - full garbage collections took 10% CPU time recently (threshold: 10%)
...
distributed.core - INFO - Event loop was unresponsive in Scheduler for 7.07s.  This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
...
distributed.comm.tcp - INFO - Connection closed before handshake completed
...
distributed.scheduler - INFO - Close client connection: Client-worker-3c6d8642-53b5-11eb-800e-32b98c347770

When I scale the job down to 250 nodes and 3 threads per worker, I still get 100% utilization, but it is slightly more stable. where the only warning messages is

distributed.utils_perf - WARNING - full garbage collections took 11% CPU time recently (threshold: 10%)

Thanks for any advice!

@Marvin How to set Kubernetes Job resource limits on Prefect3

Does anyone have a complete working example on how to use PrefectResult? Its simple: Scrape a website for a link, if the link is different than the prefect result, download it and save the new link as the result. Otherwise exit with "success: no work to be done". I don't want this to be a local result because we plan to move these flows off of a windows agent and into docker agent. Thanks in advance!

is there a way programatically to get a list of deployments in prefect? i know we can do

prefect deployment ls

but i want to get all my created deployments...1000 of them and run them all. so im assuming i have to programatically execute prefect deployment run deploymenet_name for each deployment in my deployments

@Marvin how can I automatically retry Crashed flow

@Marvin what are recommended specs for runing a local prefect server, with4-5 workpools wach with 2 workers

Good morning @Marvin - I've seen this error twice now. The deployment finishes all tasks successfully, but the deployment is marked failed for this reason:

Concurrency lease renewal failed - slots are no longer reserved. Terminating execution to prevent over-allocation.

The first time I saw this there was an outage on the prefect status page having to do with concurrency so I thought it was related to this. But this happened last night and the status page shows nothing. What is this?

Hi there, I noticed that the schedule option has disappeared from the UI Deployment page. After Friday's updates, flows do not run on schedule. Is it a bug, or you move this option to a different place? The schedule will be deprecated in Sep 2024. Also, how can we be aware of the changes you make in advance? Thanks