@Marvin How do I set a Flow to the

Failed

state from within a Task that's been submitted to the

ConcurrentTaskRunner

?

@Marvin How do I run an async flow when deploying using a prefect.yaml file

@Marvin how do i run tasks asynchronously in prefect 3.x

Hi, in prefect 2.0 - is there a way to perform 'yields' - we have some (unfortunately) long running tasks where we stream some content, which takes a while (+30 min, very large zipped json files) - and I can't seem to figure out if I can yield partial result futures in my task incrementally and start processing them in down stream tasks. Currently, the work flow is: • Stream read big annoying file into small chunks, save them in S3 incrementally in a separate thread inside the task, and return list of all the chunked file URI's as single result (= single

prefectfuture[list[str]])

object) ◦ this is where I would like to yield • Run some down stream processing on each chunk ◦ Here we use future.result() to loop over and start each task - this works 100%, but if we could use yield in the first task, we could begin processing immediately Is something like this supported / going to be supported? I tried making it work with contextmanager and tasks, but ran into nothing but trouble

Is there currently a way of overriding a task name when calling it in Prefect 2? The following (which is what the documentation recommends, unless I'm reading it wrong) appears to have no effect: my_task_function(name="Custom Task Name") The task run name for the above task ends up being "my_task_function-6971d82e-1"

@Marvin what causes the database to be locked? I find this to happen a lot when I run many flows in parallel

The concept of Agent and Work Queues is really useful. But as far as I understand, Work Queue's hold only flows, not tasks. What queue do tasks go into? Is there a way for a user to tap into that queue?

I'm running into some memory limits with a flow using an ECSRun config on a LocalCluster DaskExecutor. The first task in the flow is to generate a list of file keys, and then I map over that list to process the files. I would have expected that when an individual Mapped Child task completes, any memory used in that task would have been released (aside from the return value). But it seems like that is not the case, as the process eventually runs out of memory after several Mapped Children finish. Is there something I can do in the configuration of the flow or task to mitigate this? As an aside, when I run this same flow using a FargateCluster, it will complete without issue. I'm not sure if that's because the FargateCluster I'm using just has more memory, or if there's something inherently different between the two different DaskExecutor classes. Thanks as always for any help you can provide.

Hi im getting error when runing

prefect worker start --pool my-docker-pool

" Unable to start worker. Please ensure you have the necessary dependencies installed to run your desired worker type." can someone please help . Thanks

What is the proper way to stop local Prefect? I run 'prefect server start' as backend server and then press Ctrl+C to stop it, next time you start it Prefect throws a lot of duplicate violation errors and doesn't start properly (you can't create a project for example)