is there anyway to set the

max_retries

on a task from the UI (or without having to upload new code)? I was hoping I could template the value and pass it to the @task decorator, but it doesn't look like you can use templates with the

max_retries

argument

Hey all, I'm having some issues where flow runs will hang indefinitely. There seem to be two issues: 1. Root cause of why sometimes a flow will just run indefinitely and stop logging after some amount of time 2. Flow not timing out even though I've configured the flow with

@flow(timeout_seconds=16200)

for a 4.5 hour timeout. I'm mostly trying to solve the second one here. I see that some flows do actually hit this timeout and others will run for 9 hours and still not hit it. In the documentation I see a note "Flow execution may continue until the next task is called", and I'm wondering if maybe this could be causing my issue? I'm not actually using tasks per se, but I do notice that in the flows that do not time out there are no logs for hours vs the ones that do time out, there are logs right up until the timeout. Ironically that's the opposite of the behavior I want, which would be timing out iff there is no activity for a long time, but that's neither here nor there. Any help would be much appreciated!

What’s the best practice for Data Retention Policy on Prefect deployment runs? Just as a reference, here is how it is implemented for Apache Airflow, as yet another garbage collector DAG: https://stackoverflow.com/questions/66580751/configure-logging-retention-policy-for-apache-airflow I’m sure that Prefect has either a built-in mechanism for that, or encourages a common idiom for rotating / archiving / deleting artifacts from old runs. Context: We have a persistent storage on Azure Blob Storage (the S3 equivalent) where we store artifacts (e.g. output files and images) from a Machine Learning (Kedro) run. The space can pile up pretty quickly across runs and we would run out of storage, rendering our Prefect deployments not operational. What kind of policies are recommended to evict data from old runs? I don’t want to run out of space and I want the Prefect pipelines to remain operational. I know that some of you would say: “_It depends_”, so for the sake of this example let’s imagine that I have a dedicated 256GB of storage. Should I set a threshold (e.g. 70% of full) that will be as a trigger for evicting (removing) artifacts from old runs? Also, when should this run? as the first (prerequisite) subflow in my bigger flow, or as yet another deployment in Prefect on a recurring schedule? Thanks!

Something I am struggling to get my head around is security in regards to

FargateCluster

. It seems like the cluster is being assigned a public IP address. Ideally I wouldn’t want that to be the case as I don’t want people snooping on my cluster / submitting jobs. However, when I pass

"fargate_private_ip": True

to

cluster_kwargs

my cluster fails to start with the error:

Cluster failed to start: Timed out trying to connect to <tcp://10.0.1.111:8786> after 30 s

That makes sense. Someone somewhere failed to connect to a local IP address, presumably from outside the subnet. What I don’t understand is how I can prevent people from arbitrarily accessing my cluster from the internet whilst allowing all the ‘right’ traffic through

Hello people, anybody know how to display progress bars from a running task on the prefect URL UI in logs

Hey folks! Does anyone have an example repo they could share that's handling the deployment of a Prefect Agent with AKS? Currently starting off with spinning up a cluster with Terraform, but my k8s skills are sub-par so some useful starters would be handy. For reference i've CDK'd up a Prefect Agent & respective cluster in ECS on AWS before, but that wasn't involving k8s.

@Marvin how do I set the number of concurrent flows for a deployment? if I don't specify one, is the default 1?

Hi @Marvin I want to run a flow from another flow, I am able to do it But the catch is, my main flow should not wait until the child flow finishes, it needs to show the status as completed/ success/fail . After it triggers the child flow. I can't use automations option as I need to trigger child only in few conditions of the parameters passed from deployment. How can this be achieved any guidance please

Hi all, just getting started with Prefect and I'm stuck on error: ValueError: No Prefect API URL provided. Please set PREFECT_API_URL to the address of a running Prefect server. prefect version = 3.1.1 I've confirmed the server is running at http://127.0.0.1:4200 I've updated pyproject.toml with

#  pyproject.tom

[tool.prefect]
logging.level = "DEBUG"
api.url = "<http://127.0.0.1:4200/api>"

Any guidance is appreciated

@Marvin Hello Can you help on how we can connect to MS SQL Server Database..