@Marvin or whoever: I'm trying to build a pretty simple api that takes an audio url, transcribes it, runs some post-processing on it, and returns it. Using prefect has been a real uphill battle. Even the example repo for FastAPI was broken, none of the docker-compose files worked without fixing the async await code. when I finally got some hello world code to work at all, it fails every 2nd or 3rd request. Typically, when I experience such hardship with a particular technology, I chalk it up to, "I'm probably not using this for its intended purpose." In this case, this is what it's for, no? Is this the correct tool for just "do a pretty simple workflow, respond with the answer?" Was the background .delay() functionality just bolted on to look like celery? What am I missing?

Sounds like you've had quite the adventure with Prefect. It's always a bit of a journey when things don't work quite as expected, isn't it? Let's break down some of your points and see if we can make this a little less of a trek through the unknown. 1. Purpose of Prefect: Prefect is primarily designed for orchestrating complex workflows, primarily in a data engineering context. However, it should also be capable of handling simpler workflows like the one you're describing. That said, it might be overkill if your main goal is simply handling API requests synchronously and returning responses. Traditionally, Prefect is better suited for handling asynchronous tasks that can benefit from scheduling, retries, and monitoring. 2. FastAPI Example Issues: It's unfortunate that the FastAPI example and docker-compose files gave you trouble. Prefect's examples can sometimes lag behind new releases or changes. But they are indeed supposed to aid in understanding how Prefect can be used in a typical setup. 3. Error Frequency: If your code is failing every few requests, it may be due to resource limits (like memory or connections), concurrency issues, or something not properly async-safe. Prefect tries to manage tasks efficiently, but when integrated with FastAPI, scoping the async logic correctly becomes crucial. 4. .delay() Comparison to Celery: Prefect indeed supports running tasks asynchronously, but it's structured differently from Celery. Prefect's focus on orchestration might make it seem a bit cumbersome compared to the task-focused design of Celery. Prefect's async functionality isn't merely bolted on; it's a core part of its design, but with a different focus than Celery's. Before we jump further into conclusions, let's see if there's any specific documentation or common issues related to your concerns. I'll take a look into both the Prefect documentation and recent GitHub issues to see if any known problems could be related to your experience, especially with FastAPI and Docker setups. Let's also see if there are any specific insights into common pitfalls. It seems you've stumbled upon some known issues with Prefect's integration around FastAPI and Docker compositions. Let's unpack what was discovered: 1. Documentation Search Error: There was an issue during my search for the Prefect 3.x documentation. However, based on the context, Prefect's integration with tools like FastAPI can be tricky, especially when asynchronous handling is involved. 2. Known Issues with Prefect and Docker: - There is a reported issue related to using Prefect with PostgreSQL databases, which could potentially cause errors (like "Integer exceeds 64-bit range") in certain database configurations due to serialization challenges. - Another report highlights an error (

httpx.LocalProtocolError

) that occurs sporadically, suggesting issues with handling connections in an HTTP2 environment. This might resonate with your problem of intermittent failures. - Some users have also had challenges when running Prefect flows from frameworks like Flask, with issues such as

MissingContextError: No settings context found

. Although this problem was noted with Flask, similar async context issues could occur with FastAPI. 3. Stability and Configuration: Given the errors you've experienced, I'd recommend revisiting the configuration and ensuring you're using compatible library versions for async operations. The Docker setup can be particularly sensitive to resource allocation and networking configurations which might lead to inconsistently handled requests as you've experienced.