@Marvin I get this error when running a prefect flow, what could it be?

RuntimeError: Failed to clone repository '<https://git.myorg/project/repo.git>' with exit code 128.

@Marvin How can I setup Prefect server 3.x using docker-compose

Hey folks, I'm looking into migrating a django + celery app to use Prefect for async task processing – we run X celery workers, which all listen to a queue broker (redis/rabbitmq/etc), and our API adds jobs to the queue as users trigger events. Pretty standard stuff. I'm trying to recreate a basic version of this in Prefect. I've got a prefect server running; and several workers running in docker instances, each joining a work pool. The workers have the application code baked into the image. In celery-land, I'd just trigger jobs by calling

my_decorated_func.apply_async([args])

, i.e. `say_hello_world.apply_async(["Marvin"])`; and the workers would pick up the jobs, set up app internals (environment config et al), and then run the decorated function automatically. I'm not seeing an obvious way to do this with Prefect. I can call my

say_hello_world

flow directly, and it'll run locally, but I need it to run in the worker pool. Calling

.deploy()

tries to register it with the default worker pool, which is great, but it complains about needing an entrypoint or image. I saw some comments online about using 'local storage' to point to the specific file the flow is in, i.e.

/path/to/file/flow.py:say_hello_world

, but... there's no way that's the "right" way to queue a job, right? I get that the Prefect control plane allows for total independence between the place that's queueing jobs and the place that's executing them, but in my case, they're both the same docker image; just with different entrypoints (starting the API vs starting the prefect workers). What's a clean way to just say "look for this exact same decorated function in the worker", essentially as if it were running locally but in a different container? CC @Marvin

@Marvin how to refactor a code that uses asyn..await syntax to work with Prefect?

@Marvin how can I automatically retry Crashed flow

Built Docker image not used by my worker (/opt/prefect/my_flows not found by worker, but findable locally in container) Hi all ! Hope you're doing well ! I'm currently stuck deploying my flows on my GKE cluster. Here is my configuration : I'm using

prefect deploy

from my

my_flows

directory containing my

prefect.yaml

file

name: my_flows
prefect-version: 2.11.3

build:
  - prefect.deployments.steps.run_shell_script:
      id: get-commit-hash
      script: git rev-parse --short HEAD

  - prefect_docker.deployments.steps.build_docker_image:
      id: build-image
      requires: prefect-docker>=0.3.0
      image_name: my_repo/my_flows
      tag: "{{ get-commit-hash.stdout }}"

push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker
      image_name: "{{ build-image.image_name }}"
      tag: "{{ build-image.tag }}"
      credentials: "{{ prefect.blocks.docker-registry.docker-registry }}"

pull:
- prefect.projects.steps.set_working_directory:
    directory: /opt/prefect/my_flows

deployments:
  - name: process-lyrics-deployment
    version:
    tags: ["process-lyrics"]
    description:
    schedule: {}
    flow_name:
    entrypoint: process_lyrics/main.py:process_lyrics
    parameters:

    work_pool:
      name: pool-1
      work_queue_name:
      job_variables:
        image: "{{ image_name }}"

And this dockerfile

FROM prefecthq/prefect:2.11.3-python3.8

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . /opt/prefect/my_flows/

When I check my container locally, I can find everything in the directory

/opt/prefect/my_flows/

(as wanted) But when I run my job on my cluster I get this error...

FileNotFoundError: [Errno 2] No such file or directory: '/opt/prefect/my_flows'

I'm using one single worker, with the same prefect version as my CLI (

prefect:2.11.3-python3.8

), and configured with

pullPolicy: Always

Since I've seen this forum post : https://discourse.prefect.io/t/how-to-configure-the-pull-step-for-a-project-building-a-docker-image/2777/2 But it doesn't seem to work for me... Thanks in advance for the help 🙏

(prefect 2.0) We're reliably seeing 503 errors when trying to delete deployments. I suspect that this is happening bc these deployments have many flow runs associated with them. debug info in 🧵

@Marvin how to disable task caching system

what's the recommended concurrency limit per work queue and per work pool when using the default task runner?

Hi, when I run Prefect server with my custom docker image, I get the following error

Failed to load and execute flow run: ModuleNotFoundError("No module named 'pandas'")

I don’t think it’s coming from missing pandas module. When I log into my docker image, I have no problem importing pandas. I’ve installed all the required packages in the base environment. I’ve also tried loading all the packages in the conda environment and that the conda environment is loaded upon docker image is loaded. Is this some python path issue? What I don’t understand is that i’ve done this several times before and i didn’t have this issue until now