I assume this has been mentioned before, but is there a way to get prefect's logging to play nice with python's

logging

library (or stdout/stderr)? We don't want the prefect dependency and get-flow-logger calls in all of our business logic code, but it'd be nice to see debug and error code in the prefect cloud console

What is the Prefect 2 equivalent of this? task_args={"name": "Get Data Part One"}. Context: I have 1 task that I use multiple times in the final flow run / final task, and I need to differentiate each one for the logs.

When using prefect config set PREFECT_API_URL="http://192.168.128.58:4200/api", are there any other settings I need to set to match this? I'm struggling to connect to the prefect UI remotely

Hi Team, What is the way to install a "com.crealyticsspark excel 2.120.14.0" to cluster at runtime in prefect2

Hey Folks 👋 Lately, some of my flow runs have a "Run Count" of 2 (instead of 1). When looking at the "Results" tab I see that both runs were started pretty much at the same time. We're using k8s infra. What could be the conditions of a single flow run being started more than once? Having a flow launched twice can be quite problematic, especially when they have the same id.

Hi everyone, has anyone experienced when using Dask executor freezes from transitioning from completed task to the next. Over the last few days one of my largest flow keeps some tasks in ‘running’ state way pass timeout and does not want to notify the server that it has to be restarted. I had 2.10.4 yesterday and 2.10.8 today but this hasn’t improved it.

Hey folks! I have a simple question as we do our migration to V2. Can the prefect agent still run in docker?

Hello everyone, we would like to use a template for our deployments. This template takes the necessary information from a JSON file in the same directory. As I understand it, I have to achieve uniqueness via

path

and

entrypoint

so that the central template becomes unique deployments in remote storage. Unfortunately, I am currently failing with FileNotFoundError. Example deployment and error message can be found in the thread. What am I missing/doing wrong? Thanks in advance for any hints!

Hi everyone! I'm just exploring prefect in CLI, and I can't make the prefect work pool follow the cron schedule (it just runs once and that's it). Hoping someone can help, thanks in advance! (my dockerfile contents in replies)

Hi all, anyone know how I can set a work queue’s priority with the cli?

Hi, has anyone successfully run connect prefect with an existing static dask cluster in kubernetes? I have setup a local dask cluster using the official dask helm chart with minikube and did port-forwarding so that prefect on my local machine can access the scheduler in the cluster. I can see that this connection was successful and the workers are receiving the task. But this is the error message:

Exception: PrefectHTTPStatusError("Client error '404 Not Found' for url '<http://ephemeral-prefect/api/task_runs/4d756c49-c449-4729-94a9-a7729d78e160/set_state>'\nResponse: {'exception_message': 'Task run with id 4d756c49-c449-4729-94a9-a7729d78e160 not found'}\nFor more information check: <https://httpstatuses.com/404>")

Hello everyone, quick question about infra blocks vs workers, Is there an equivalent to

infra_overrides

in workers ? We use a default config in an

ECSTask

and override with the docker image we want to use, Not sure how to have the same with workers ?

Hi all, I'm working on our migration from Prefect 1 to 2, especially for the deployment. With Prefect 1, I build my Docker image, push to Artifact Registry in GCP, then I register my flow to Prefect Cloud (and adding some parameters like CronSchedule, Dask runner, etc...). Is this always a good way to deploy a flow with Prefect 2? I can't find any example with a private GCP registry for Docker image.

Hey guys, I would like to instrument my self hosted Prefect deployment with

OpenTelemetry

(Its the last part of our micro-service architecture thats missing it) I was looking online and only managed to find a single old thread and one attempt that doesn't look working/maintained. Anyone tried this? Know if its doable even? • It would require passing the Open Telemetry trace id from a parent flow to each of its subflows/tasks. Prefect run context which might be an option for this does not appear to allow additional fields. • The tasks/flows themselves would also need to be instrumented of course Would appreciate any help 🙏 Thanks all

Are there any docs on sizing and Prefect 2.0 Open source server?

@prefect Would like to know in prefect2

signals

is changed to which supported format?I need to raise exceptions failed states.Please do provide the needful info

Hey all - I have 2 prefect agents (1 on ECS, 1 on an EC2). I want a deployment to run on the ECS task after the EC2 flow finishes (orchestrator pattern) From my googling, it seems that there are two options: • use an automation (limited to 3 per workspace) • Use a custom script (like here) Has anyone come across a simpler, more scalable way to do this? I'm hoping to avoid adding a bunch of code to my flows just to make a handoff between infras, if possible

Hi, I use Dask executor on relatively large flow on local agent and over last few days after hour or so it stops updating the status. The flow is in state Running, but there aren't any running tasks. Some are pending, but no updates in logs. Has anyone experience that? Is it possible there is some keepalive timeout setting I need to update?

For the past week we've been unable to see important pieces of our cloud 1.0 dashboard, but only for one of our tenants. Is there a reason this might be happening?

Hey guys, I have a streaming pipeline that I built based on this git repo: https://github.com/anna-geller/prefect-streaming I'm running it on a fairly large EC2 instance (2xlarge) and after running for a couple of hours; the instance is dying due to out of memory issues. The task is primarily setup and defined as such:

bumping this if anyone has any idea https://prefect-community.slack.com/archives/CL09KU1K7/p1683898273303989

Hey all, I'm trying to get a test deployment setup using a private github repo as my storage block and a docker image for infrastructure. When trying to build a deployment with

prefect deployment build ./myflow.py:my_flow -n docker-testing -sb github/test-repo -ib docker-container/testing

I get the error `RuntimeError: Unable to load 'testing' of block type None due to failed validation. To load without validation, try loading again with

validate=False

.` I created the docker container block in the UI and I'm not sure what kind of "validation" I'm missing (or why it says of block type

None

). Any advice on how to get past this? EDIT: I am using Prefect Cloud btw

Hi all do you guys have some good tutorials on howw to run apache spark with prefect? can you guiys point me to some good tutorials of resources?

What's a good practice for handling subflow failure? For e.g. I have a case of a flow triggering many (e.g. 15) subflows and one subflow fails.

If I have a flow running every hour but something causes the flow to take more than an hour, can Prefect delay or cancel an execution so that they are not overlapping?

The Prefect base image doesn’t pass Snyk Docker vulnerabilities check, any ideas? When scanning the base image with Snyk I find this vulnerability:

✗ Critical severity vulnerability found in curl/libcurl3-gnutls
  Description: Cleartext Transmission of Sensitive Information
  Info: <https://security.snyk.io/vuln/SNYK-DEBIAN11-CURL-3320493>
  Introduced through: git@1:2.30.2-1+deb11u2
  From: git@1:2.30.2-1+deb11u2 > curl/libcurl3-gnutls@7.74.0-1.3+deb11u7

This comes from Prefect’s Dockerfile line 101: https://github.com/PrefectHQ/prefect/blob/main/Dockerfile#L101 Is Git actually needed and used within the Prefect server / Prefect agents? If so, why? Can I remove it or will it break Prefect?

hi everyone, anyone know how to schedule cron jobs on UI now? I’m not seeing deployments in my prefect cloud as well.

HI there, my flow stuck at :

| INFO    | Flow run 'dJhUY-64621fc68e5d1ff93022ff8a' - Downloading flow code from storage at '/usr/src/app'

I know there is not any large files. and there is no error , the agent only output this error. Any help on how to debug it ?

If I want to run my flows within a docker-container block but I have prefect-snowflake, and prefect-dbt as dependencies. Is there a better option than editing the ENV value to install them with EXTRA_PIP_PACKAGES? When I do that it looks like it keeps re-running the PIP command each time the docker image spins up.

Hi Everyone, can anyone please tell why i am getting this error.

Crash detected! Execution was interrupted by an unexpected exception: httpx.HTTPStatusError: Client error '404 Not Found' for url '<http://orion-url/api/task_runs/24fd3ed1-78e5-49b2-84ab-ea42c385c942/set_state>'
For more information check: <https://httpstatuses.com/404>

I stated inference in 500 inputs, and after sometime it gets crashed. some of the ques i have is. 1. Why it gets crashed, is it because I gave 500 inputs and prefect server fail to handle it. 2. If it gets crashed, is there any mechanism to handle this. 3. And I am using Kubernetes as infrastructure Can anyone help me please