@Marvin how can i use prefect with django?

Hello 👋 Is there a way to retry

CRASHED

tasks or to avoid setting them as

CRASHED

? This issue was resolved and we can have Dask to retry

RUNNING

tasks now. But since the prefect

2.8.7

version, prefect detects that the Dask worker will be shut down and sets the task as

CRASHED

. When Dask resend the failing task, we end up with

Task run '...' already finished

.

Hi All, I want to schedule a prefect flow every minute and within the flow have a check to determine a file in s3 has changed, if the file is changed a bunch of tasks are executed, otherwise flow exits, for this I need to maintain state across flows, how can I do that, thanks a ton

Hey all, i'm struggling to connect to our Azure Devops Repo using the GIT storage class. I keep getting an HTTP Unauthorized error. I've tried connecting with git credentials and with a PAT and neither work. Has anyone else managed to connect to AZ Devops?

@Marvin How to set the

PREFECT_API_URL

in a

prefect.toml

configuration file?

@Marvin how do I add a timezone to a cron expression when deploying a flow with the flow.serve method?

@Marvin why would a server fail to start with this error:

sqlalchemy.exc.DBAPIError: (sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.DeadlockDetectedError'>: deadlock detected

Hi , is there a way to deploy multiple data pipelines(one for daily and other pipeline for weekly etc) , would appreciate the help , thanks in advance

Hi, I am using prefect to run multiple spark jobs on k8s. Each of these jobs can run long running with some of them have execution times of more than an hour. The jobs are mapped and executed. However in some instances I can see that only few of the jobs in the map are executing. I am using

LocalExecutor

which these mapped jobs are running sequentially. Are there instances where the LocalExecutor is not able to track the spark job running on k8s and thus not able to trigger the subsequent tasks in the map ?

Hey all, I'm having some issues where flow runs will hang indefinitely. There seem to be two issues: 1. Root cause of why sometimes a flow will just run indefinitely and stop logging after some amount of time 2. Flow not timing out even though I've configured the flow with

@flow(timeout_seconds=16200)

for a 4.5 hour timeout. I'm mostly trying to solve the second one here. I see that some flows do actually hit this timeout and others will run for 9 hours and still not hit it. In the documentation I see a note "Flow execution may continue until the next task is called", and I'm wondering if maybe this could be causing my issue? I'm not actually using tasks per se, but I do notice that in the flows that do not time out there are no logs for hours vs the ones that do time out, there are logs right up until the timeout. Ironically that's the opposite of the behavior I want, which would be timing out iff there is no activity for a long time, but that's neither here nor there. Any help would be much appreciated!