Hi! We use prefect to run databricks tasks, and we've been noticing lately that the status prefect shows for a task doesn't always update in time - prefect frequently thinks something is still running, or hasn't even started, when it already started and finished. So the flow doesn't continue on to the next tasks in the dag and gets stuck. Eventually gets the updated status of the databricks tasks and continues. This seems like some sort of communication issue between prefect and databricks, but I can't figure out how to debug. Would appreciate some help here! Thanks!

Hi @Marvin. I'm trying to deploy a simple flow that has access to the local file system that the worker pool is running on. using flow.deploy() it builds an image and runs nicely but I can't figure out how to mount local volumes to the container. how can I do this?

@Marvin How to Clean the prefect salute DB data ?

Hello here, I am new to Prefect and run my first flow in my local K8S cluster and use

KubernetesRun

. The flow is trying to download the

py

file from a local git repo in the same local K8S cluster. But I got an

SSL_VERIFICATION

error even though the

url/host

for

git storage

is

HTTP

. Is there anyone having similar issue?

I'm creating an integration with datadog and I need to add a custom handler to prefect's flows. I need to add a handler that gets all logs from the run and spits it out as json. I've seen a lot of examples like the one bellow, but that's not what I need. I don't want to add a handler to each logger.

logger = prefect.context.get('logger')
logger.addHandler(log_handler)

I need something like this

log_handler = logging.StreamHandler()
log_handler.setFormatter(DatadogFormatter())
get_logger().addHandler(log_handler)

I add the handler to the 'root' logger and everything is logged the way I need. I've tried the latter and it works fine with

flow.run()

, but when I register the flow I can't get it to work with k8s. More details in the thread

Hello all, is creating a custom webhook supported on the local version of the server? It lets me create the webhook, but all the fields (e.g JSON data) are all disabled and so when the webhook is called none of the fields are passed.

@Marvin how to write tqdm output to prefect logs

Hello everybody! I've been playing around with the idea of self hosting Prefect server on a VM instance using docker compose. The following posts have gotten me pretty much all the way to having a working self-host option: • https://dev.to/calvinqc/the-easiest-docker-docker-compose-setup-on-compute-engine-1op1 • https://medium.com/the-prefect-blog/how-to-self-host-prefect-orion-with-postgres-using-docker-compose-631c41ab8a9f I am able to just set my local

PREFECT_API_URL

env var to the external IP address of my vm instance (not the nicest solution but I can clean this up later) and then I can immediately push my blocks and deployments to the server! 👍 The problem comes from when I lauch a run of one of my test deployments that use a Cloud Run Job. I never get out of the "Pending" status in the Prefect UI. Pretty sure it is something with my networking and probably not an actual issue with Prefect, but I can't seem to make any progress. I've tried setting up Serverless VPC Connections to allow the Cloud Run Job to be able to connect to the vm that has the server running, but maybe I've set it up incorrectly 🤷. Anyone else ever managed to get a cloud run job deployment to work with a vm instance hosted prefect server?

I am using PREFECT_LOGGING_EXTRA_LOGGERS to set an extra logger from one of my libraries to show up in my flow logs. I want to set the log level of that extra logger. mylib's logger has level set to DEBUG because that is needed by one of the two handlers mylib.logger's FileHandler has level set to DEBUG mylib.logger's RichHandler has level set to INFO I want the output of the logger in PrefectUI to also be INFO but I'm not sure how to set that. I'm currently seeing all the DEBUG messages, which is too much

@Marvin is there a way to get my flow state using flow id?