We’ve noticed that Prefect is sometimes unable to clean up after flows or tasks that fail unexpectedly without allowing Prefect exception hooks to run (for example, if a debugged local run is abruptly exited, or if a flow run crashes on infrastructure). This can leave “zombie” flows that show as “running” in the UI for days when in reality, they stopped a long time ago. This also may occupy a task concurrency slot that will never be freed up because the task exited abruptly before cleanup code executed. We need a way to clean up the flows and task concurrency slots that are left as zombies. Anyone from Prefect have a cleanup flow? Does anyone have some cleanup code they are willing to share? CC: @Marvin

Prior to prefect, I have used AWS Step Functions to manage pipelines of tasks. What is the comparable approach to Map, where I can take a list of inputs, and in blocks of N, send each item of N to a task (e.g. AWS Lambda function). I could loop over items one at a time, but I wasn't sure if there was a feature that I am overlooking that gave us control similar to Map?

Hi everyone! Is PREFECT_SERVER_API_HOST=0.0.0.0 still valid in Prefect 2.16.0 and later? Marvin says there is no reference in the documentation but if I do not set then Prefect will not accept requests from any IP

@Marvin can you show an example flow using a CronSchedule in its deployment?

Hey team - how does the --install-policy if-not-present pull in the proper versions? We have an ECS worker running

prefect worker start -t ecs -p ecs-worker-pool --install-policy if-not-present

Warning! Failed to load collection 'prefect_aws': ImportError: cannot import name 'deprecated_class' from 'prefect._internal.compatibility.deprecated' (/usr/local/lib/python3.10/site-packages/prefect/_internal/compatibility/deprecated.py)

Prefect’s unique value proposition - DAG visualizer, clean, easy to onboard and easy to integrate and deploy With so many tools for: 1. Task scheduling (e.g. Celery, Ray, dask, bullmq) and 2. Workflow/DAG scheduling (e.g. Apache Airflow and Prefect), one can get real puzzled on the workloads / use cases / applications. It’s hard to put Prefect on the map. Personally, it took me so much time to understand that Prefect is most suitable for data pipelines (e.g. ETLs, Data Science / ML pipelines) due to its: 1. DAG Visualization 2. Fine-grained per task configuration (a node in the DAG) by using decorators. While you could chain tasks using Celery and its web UI, it doesn’t come close to Prefect. 3. Development and production integration a. The most notable feature of Prefect is its integration (not really an integration per se but ability to easily get the YAML is something) to

docker compose

and to production environments, i.e. Kubernetes. b. Integrating the Helm Chart of Prefect was a breeze and we have it in production. c. As well as its ability to scale (we don’t use that currently). @justabill do you have collaterals for Prefect that shows where Prefect is in comparison to other/similar task/workflow schedulers?

Does prefect still use

pg_trgm

and how is it used? All the starting guides just have you spin up a postgres container in docker which does not have this extension turned on?

When deploying through the CLI with --cron command.. Is there a way to add the timezone argument to this as there is when creating a deploy via python?

Hi, what is a way to pass local

.prefect/config.toml

file to runtime environment? For example, I’m writing

pytest

in VScode, when I run the test, it always fail for not finding the secrets that specified in that config file. I can only run in CLI

python flow.py

fine.

How do I get

CronSchedule("0 11 * * *")

to fire at 11 and not 12?