Second question - we use django's ORM for db models. Are there any issues with passing these models between tasks? I think they're serializable.

Hello, I was wondering if it would be better to start the worker from the gitlab ci or using system service?

Hi everyone, we are running prefect3 on k8s and for some automations we have a time delay of 15minutes. Automation is to send a notification if a flow run enters Completed status. Sometimes the automation is immediate but most of the time there is a 15min delay. Any ideas what this could be?

I'm seeing variables change within a mapped task where I have the task call a function. The function prints out the row mapped by the dataframe and returns that value, but later uses of the variable have different values than what was set by the function previously. Did something change from v2 to v3? Should I make the function a task to have it behave normally?

hey, nothing that I'm aware of. Could you share your account ID and more specifically what you're seeing?

Hi everyone, I am trying to run ECS Tasks on AWS. I setup a self-hosted Prefect server, and a worker service that submits tasks to ECS. However, these tasks are not running correctly. These tasks are Docker containers that I have deployed on the self-hosted prefect server (and I run them on there). I think the main problem is that Fargate is running on x86_64 and these are ARM. I can't figure out how to get ARM running on Fargate and if I switch the built to x86_64/AMD, then the container isn't pulled correctly from ECS. I am not sure how to remedy this and would like some help in figuring it out. I have pasted the details in the comment on the yaml/code. I have tried editing the base worker template and nothing happens. I decided to then create a custom task definition and now instead of getting the failure

exec /usr/bin/tini: exec format error

I got the PREFECT in ASCII and then nothing which I imagine is because the Prefect Task/RunId is invalid? I'm not sure what to make of the entire thing and I am at a loss of what to do from here.

Self hosted, v 3.4.25, Kubernetes worker Concurrency limit set to 1 at deployment level, policy set to cancel new Common behavior: duplicate runs start, several seconds apart, one gets cancelled -- Why, has anyone seen this? Example yaml deployment config:

- name: other
  description: XXX
  entrypoint: XXXX:build_other
  schedules:
      - cron: "0 6 * * *"
        timezone: "Europe/Prague"
        active: true
  parameters: {}
  work_pool: *transformations
  pull: *work_dir
  concurrency_limit:
    limit: 1
    collision_strategy: "CANCEL_NEW"

Hello all, I am doing Gitlab migration with my current private repo using Prefect Cloud. However I am struggling with the pull step using the PAT (Personal or Project Access Token). As I understand based on the code in

prefect/runner/storage.py

I have to setup the token as

oauth2:<token>@gitlab.corp-name/group/project.git

However this does not work and i keep getting error 128. Any idea how to resolve this one? Or maybe some workaround that does not depends on pulling perhaps ?

Hello. Is there any way to set up retention for Flow Runs in Prefect OSS? And also something for other tables in Postgres, such as

log

etc. We found e.g. the

PREFECT_EVENTS_RETENTION_PERIOD="2d"

for

events

table. But is there any good "Overall Retention" setup guide?

Hi all - the new UI/UX for Prefect is entirely unusable. It makes zero sense. Is there any way to roll it back?

Hi all, my company has been using prefect using a cloud run work pool for nearly 1 year now. I was investigating push work pools, so I deleted my old cloud run work pool temporarily (since free tier accounts have a max of 1 allowed). Then when I went to create a new work pool I am only able to use prefect’s managed work pool without upgrading?! The managed work pool doesn’t even allow for custom docker images. I was not clear that deleting my old cloud run work pool would regress the amount of functionality I could have on my free tier account! Please could someone advise what my options are here? Besides upgrading to pay 100$ / month?

Has anyone been having issues with pulls_from_block on github? Since yesterdays github outage i’ve been having

Failed to pull from remote:
 Cloning into '/tmp/tmp0h4b39iiprefect'...
fatal: unable to access '<https://github.com/org/repo.git/>': Failed to connect to <http://github.com|github.com> port 443 after 131017 ms: Couldn't connect to server

errors on most of my deployments

Hi all, has anyone been having issues with prefect cloud automation? Usually during midnight, it fails to report/detect the failed deployments/flows. Parent flow and child flow failed at the same timestamp. Parent flows have

prod

tag but child flows not. anyone with the same issue? during daytime its fine. super weird.

Hi, I just signed up for a hobby tier Prefect plan to try things out but can't seem to create even one workspace (which I think is required for me to do anything). What am I missing?

Hi all, why can't I add my github credentials to my prefect block? Every time I try to, it just says None in the github repository block??

Is the

prefecthq/prefect:3-latest

docker image using the latest UX? It looks different from the prefect cloud UX, but I could be wrong

Hi! I want to move my pool worker to Kubernetes, but it seems the free tier doesn’t allow adding a worker pool. I started using Prefect two years ago and it was free back then. If I delete my current worker pool, will I be unable to create a new one?

Hi Team - I signed up to evaluate perfect with a local worker. Didn't do anything, except to deploy one sample deployment and now it says "Cannot cancel subscription" . Everything is under the limit of starter pack. Subscription is a subscription and should be able to cancel and not forcing anything for a customer , right?

Morning, I've contacted support already, but is anyone else who uses cloud run workers seeing infrastructure errors due to prefect creating 3-4

CreateJob

requests in a row? Results in a 409 error for us, appears like so in logs:

Hello! the service started deteriorating in Prefect Cloud, for instance, we woke up to 10 crashed flows because what seems to be a rate limiting rule

Failed to generate job configuration: Client error '429 Too Many Requests' for url '<http://orion-internal:4200/api/accounts>...
For more information check: <https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429>

a few more also were stuck at the pending state.. is this sort of rate limiting documented somewhere for the Pro Plan? is it something we can control?

Can 'Secret' block types still be accessed in 3.6.2? I can see the secrets on the website ux, but 'prefect block ls' and python code using Secret.load('[Secret_name]') can't seem to find secrets.

Hi ? How do we set minimum resources for a prefect deployment ? I have kubernetes workers, and I need to tell them how much memory my script need to be able to autoscale my kubernetes job when deployment is running... (in Prefect 3x btw). ChatGPT give me this but I dont see any doc about it :

deployments:
- name: my_deployment
  work_pool:
    name: my_worker_pool
    job_variables:
      pod:
        spec:
          containers:
            - name: flow
              resources:
                requests:
                  cpu: "2"
                  memory: "8Gi"
                limits:
                  cpu: "4"
                  memory: "8Gi"

Hi, would someone be able to review https://github.com/PrefectHQ/prefect/pull/19498 ? Thanks!

Hey all, we're running into this scheduling issue, breaking our production pipelines. Is there an ETA on a fix here? Folks are starting to push for moving off of Prefect due to this.

hey everyone, started getting this weird issue with my Cloud Run Jobs based work pool recently. I have a flow that submits 7 flows (they are actually standalone flows with their own infra, not actually subflows). Each flow trigger its own cloud run job with a unique (prefect generated) name etc. Still I'm getting this GCP error claiming resource already exists. It seems to happen quite randomly. As you can see some of them launch successfully and sometimes all of them do. However as you can imagine it's a big problem when some of them don't. Anyone has an idea of what it can be? When I look at Cloud run I never see the job existing at any time that would justify a duplication. However it's also probably because we have

keep_jobs

set to false so even if it's duplicating some job it's probably also deleting. We're not tracking the flow state or anything like that from python, we just "fire and forget" these flows. This is my code to launch them:

for model in collection_models:
        scenarios = model_to_scenarios.get(model, [])
        if not scenarios:
            logger.warning("No scenarios found for model %s, skipping", model)
            continue

        # Submit task for this model's subflow with model-specific name
        future = launch_model_subflow_task.with_options(
            name=f"launch-{model}",
            retries=2,
            retry_delay_seconds=60,
        ).submit(
            model=model,
            scenarios=scenarios,
            collection_limit_per_brand_model=request.collection_limit_per_brand_model,
        )
        deployment_futures.append(future)

and the definition:

flow_run = await run_deployment(  # type: ignore[misc]
            name=f"process-batch-subflow/{PREFECT_DEPLOYMENT_SUFFIX}-subflow",
            parameters={
                "model": model,
                "collection_scenarios": scenarios,
                "collection_limit_per_brand_model": collection_limit_per_brand_model,
            },
            job_variables={
                "cpu": CPU_CORES,
                "memory": "32G",
            },
            timeout=0,  # Fire-and-forget: don't wait for subflow completion (execution timeout is set on the subflow itself)
            tags=[model],
        )

@Nate, any chance you can approve the auto-bot updates for conda

pydocket-feedstock

to unblock conda

prefect-feedstock

? This will help us update our env, which would be nice! https://github.com/conda-forge/pydocket-feedstock/pulls

hi all, question about the

ProcessPoolTaskExecutor

(or really, any of the parallel task runners). Since tasks can be nested inside of tasks, what's the execution model when i

task.submit()

a task, and it calls

another_task()

inside of it, where does

another_task

run? in the process that is handling

task

? i ask because it used to be that we needed to wrap tasks in flow runs to control which task runner to use

Hey all, we're seeing weird behavior with Prefect Cloud jobs when a single flow run is picked up by two ECS tasks and both execute the flow simultaneously. The logs in Prefect Cloud then appear duplicated, and one of the ECS tasks eventually fails with exception

prefect.exceptions.MissingResult: The result was not persisted and is no longer available

We observed this issue three times, first one on Saturday, then Monday, and today. This is the first time we encountered this issue after using Prefect Cloud for over 2 years now. We're using ECS Push work pools. Does anyone experience the same issue?

Hello! I'm self-hosting Prefect with SQLite and I am getting a lot of these errors: https://gist.github.com/valleybay/bb1bd9717596826e125b0343fb20c64a. It says «database is locked». It's repeating every few minutes. Anyone else experiencing this? Have you been able to get Prefect to run smoothly on SQLite?

Hi @Marvin I am hosting Prefect OSS v3.6.3 on kubernetes. I have an automation that send a slack alert when a flow enters TimedOut, Crashed, or Failed. Yesterday I got some alerts in slack, it was working. Now when a flow crashes, nothing arrives on slack even though the automation is turned on. The slack block is working: when I run these lines locally, I get the alert:

>>> from prefect.blocks.notifications import SlackWebhook
>>> slack_webhook_block = SlackWebhook.load("slack-prefect-prod-failures-v2")
>>> slack_webhook_block.notify("Hello from Prefect!")

I create the automation locally with

prefect automation create --from-file automations.yaml

. Creating the simplest automation in the UI doesn't solve the problem. What could be the cause ? I did not changed much things since yesterday.