I see this https://discourse.prefect.io/t/how-can-i-restart-a-flow-in-prefect-2-0/1332/11 but its related to restarting Flows, not individual Tasks

Hi friends!, Maybe some repository or website can help me where I can see examples of pipelines with Prefect Community using Pandas and Scikit-Learn. I am very grateful to you!!

Hey, while deploying with helm to EKS cluster I got this error a bunch of times:

create Pod prefect-server-postgresql-0 in StatefulSet prefect-server-postgresql failed error: Internal error occurred: failed calling webhook "<http://namespace.sidecar-injector.istio.io|namespace.sidecar-injector.istio.io>": failed to call webhook: Post "<https://istiod.istio-system.svc:443/inject?timeout=10s>": service "istiod" not found

none of the pods are up and they all display this message is this a prerequisite for installing prefect-server on a cluster? thanks (link to deployment charts: https://github.com/PrefectHQ/prefect-helm)

Hi everyone, I am struggling to get an Azure devops private repository connected as a storage block. I believe I if I use git_clone_url_secret_name as a parameter it will allow me to send in a Azure Devops repos friendly url. I am not exactly sure how to use the parameter. i.e. just use a github block and pass that parameter or is there a git specific block I use? Any help/pointers much appreciated!

Hey All! I'm pretty new to Prefect and am trying to figure out how far I can go with the Slack integration. I want to format a notification with data being retrieved from Snowflake to update users on a daily basis. However, I'm having a tough time figuring out if this is possible. Has anyone tried something similar?

Would it possible if someone can show me the page that has all the Prefect CLI commands?

Hi Team, We are running like 1000 inferences per day so there might be a case where prefect DB is filled with high amount of data. I just want to know how can i prune (Not delete) the database ?

Any recommendations to developing unittests for prefect blocks?

hey! i'm using the API to run ad-hoc flow runs by deployment ID is there a way I can force the deployment ID to be a the same every time I deploy it? for instance after a server crash or a deployment deletion (trying to make usage easier for other parties inside the company )

on a different issue: i'm having the same kind of problem when trying to access prefect UI on a self hosted server behind a proxy: can this be solved in deployment? https://github.com/PrefectHQ/prefect/issues/10389

Hello.

Hosting Prefect (first time) on AWS EC2 instance. Security group allows for access to port 4200.

set PREFECT_UI_URL to "http://x.x.x.x.:4200"

doesn't answer exteranlly

still answers on the instance at http://localhost:4200

How do I get the web server to bind to the external ip address?

or, preferably, to 0.0.0.0 (any) ip address?

Hi Everyone, I am running 3 different deployment with 3 different queues. if there any constraints when all 3 queue were at peak load. we are experiencing a issue where process in "q1" in getting slower when "q2" and "q3" is running. How can i optimize this. All three queues has concurrency 10

Hey everyone! I have a couple of questions and would appreciate any insights: 1. How do you manage and run both "dev" and "prod" environments within a single workspace, especially when using the

prefect.yaml

to deploy flows? 2. Has anyone used Azure DevOps Repos to store your Prefect flow code? If yes, could you share any resources or best practices you've come across? Currently using Azure Blob Storage. Thanks in advance!

How do you guarantee idempotency in your Prefect deployments? This article touches upon the importance of idempotency in microservices architecture: https://amplication.com/blog/importance-of-idempotency-in-microservice-architectures It made me wonder how do you achieve it in DAG workflows. Do you design your DAG workflows to be idempotent and thus allow retries and handle failures gracefully? If so, how do you achieve it with Prefect? Should I use the caching decorators?

Hi, I have been trying to setup an Azure Blob filesystem and keep running into issues, with the code below and a very simple block configuration I am receiving a TypeError: argument of type 'PosixPath' is not iterable . I have the Blob container name in the storage bucket path and specified the connection string in the config. Any tips/ideas on how to debug/fix would be much appreciated!!

from prefect import flow
from prefect.filesystems import Azure

az_block = Azure.load("azure-blob-block")
az_block.get_directory()

@flow(log_prints=True)
def hi():
        print("It worked!!")

Hi, can anyone help? I am trying to start prefect with docker using the following compose file:

version: "3.3"
services:
  database:
    image: postgres:15.2-alpine
    ports:
      - 5432:5432
    volumes:
      - ./postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_DB=prefect

  server:
    image: prefecthq/prefect:2.11.5-python3.11
    restart: always
    volumes:
      - ./prefect:/root/.prefect
    entrypoint: ["/opt/prefect/entrypoint.sh", "prefect", "server", "start"]
    environment:
      # If you want to access Prefect Server from anywhere other than the Docker host machine, you will need to change
      # PREFECT_UI_URL to match the external hostname/IP used to load the  UI in your web browser.
      - PREFECT_UI_URL=<http://127.0.0.1:4200/api>
      - PREFECT_API_URL=<http://127.0.0.1:4200/api>
      - PREFECT_SERVER_API_HOST=0.0.0.0
      - PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://postgres:postgres@database:5432/prefect>
      # Uncomment the following line if you want to use the 'S3 Bucket' storage block instead of the older 'S3' storage
      # - EXTRA_PIP_PACKAGES=prefect-aws
    ports:
      - 4200:4200
    depends_on:
      - database

, but I always get this error:

prefect-docker_server_1 exited with code 1
server_1    | Traceback (most recent call last):
server_1    |   File "/usr/local/lib/python3.11/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
server_1    |     return fn(*args, **kwargs)
server_1    |            ^^^^^^^^^^^^^^^^^^^
server_1    |   File "/usr/local/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 225, in coroutine_wrapper
server_1    |     global_thread_portal = get_global_loop()
server_1    |                            ^^^^^^^^^^^^^^^^^
server_1    |   File "/usr/local/lib/python3.11/site-packages/prefect/_internal/concurrency/threads.py", line 251, in get_global_loop
server_1    |     GLOBAL_LOOP.start()
server_1    |   File "/usr/local/lib/python3.11/site-packages/prefect/_internal/concurrency/threads.py", line 148, in start
server_1    |     self.thread.start()
server_1    |   File "/usr/local/lib/python3.11/threading.py", line 957, in start
server_1    |     _start_new_thread(self._bootstrap, ())
server_1    | RuntimeError: can't start new thread
server_1    | An exception occurred.

Hello Community ! Is it possible in Automation pass some output from parent Flow to a triggered Deployment ? Or if there are any ways how I can call one Flow (or deployment) from another ? (codebase is different, so I can’t directly import files)

Hello Community ! I just started with prefect and had a doubt. I was going through the examples in the docs for Tasks caching and came across this one using the task_input_hash for the cache_key_fn. example: from datetime import timedelta from prefect import flow, task from prefect.tasks import task_input_hash @task(cache_key_fn=task_input_hash, cache_expiration=timedelta(days=1)) def hello_task(name_input): # Doing some work print("Saying hello "+ name_input) return "hello " + name_input @flow def hello_flow(name_input): hello_task(name_input) if name == "__main__": hello_flow("world") When i run the flow everything works fine as expected , i.e when i rerun it i get the cached tasks result but the issue is in the UI under the task detail section i am getting Cache Key as None as well as Cache Expiration as None. Why that so ? Am i missing something?

hey everyone, have a question - i was able to spin up prefect in a container and start the server up. i can visit the ui just fine. i also tried to create a flow, and the data populated my sqlite db:

root@3032594fbf4c:/opt/app/initial_test# python3 hello.py
21:18:58.862 | INFO    | prefect.engine - Created flow run 'impressive-shrimp' for flow 'hello-world'
21:18:58.864 | INFO    | Flow run 'impressive-shrimp' - View at <http://myapp.com/prefect/flow-runs/flow-run/6e4ca6b7-3873-4ca0-a36b-2083992a77ad>
hello world
21:18:59.004 | INFO    | Flow run 'impressive-shrimp' - Finished in state Completed()
root@3032594fbf4c:/opt/app/initial_test# python3 hello.py
21:21:11.399 | INFO    | prefect.engine - Created flow run 'victorious-raptor' for flow 'hello-world'
21:21:11.401 | INFO    | Flow run 'victorious-raptor' - View at <http://myapp.com/prefect/flow-runs/flow-run/9406a029-1952-421f-ac86-409bf2df944c>
hello world
21:21:11.545 | INFO    | Flow run 'victorious-raptor' - Finished in state Completed()

but none of this shows up in the actual webapp... when i try to visit the above it shows me 404. also i dont see any output from the server - so it's almost like prefect is running the job, but doesn't communicate it with the actual server...

prefect config set PREFECT_SERVER_API_HOST="0.0.0.0"
prefect config set PREFECT_UI_API_URL="<http://myapp.com/prefect/api>"
prefect config set PREFECT_UI_URL="<http://myapp.com/prefect>"
prefect config set PREFECT_API_DATABASE_CONNECTION_URL="sqlite+aiosqlite:////opt/app/prefect.db"

hi! im trying to use timeout_seconds function to end a task if it takes more tan 60 seconds to finish. however the timeout seconds are being ignored. someone know what can I do?

Hello all, I have a question about prefect. Currently I have an abstract factory which instantiates a bunch of products and each one of those products I want them to ideally contain a flow, but reading about how prefect works it doesn't really allow for OOP based flows? Any advice on how I can add it do our existing structure?

Hi i am receiving this error when i am running prefect on my mac, any ideas?

Traceback (most recent call last):
 File "/Library/Frameworks/Python.framework/Versions/3.10/bin/prefect", line 5, in <module>
  from prefect.cli import app
 File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/prefect/_init_.py", line 45, in <module>
  from prefect.engine import pause_flow_run, resume_flow_run
 File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/prefect/engine.py", line 98, in <module>
  from anyio import start_blocking_portal
ImportError: cannot import name 'start_blocking_portal' from 'anyio' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/anyio/_init_.py)

Newby question. I am installing version 2.11.5 in my python venv. I get the following error however.

ImportError: cannot import name 'start_blocking_portal' from 'anyio' (/home/gtracy/prefect-env/lib/python3.10/site-packages/anyio/__init__.py). I'm using Python 3.10.12.