Could someone tell me why when creating a flow schedule, Prefect creates multiple flows run for the same flow? In this case I have 1166 flows scheduled to run per hour and 3 new schedules were created for each flow. Would I be able to limit the creation of these future schedules?

Hi Team, Can anyone help me in triggering a flow whenever a api is hit. i'm getting problems with prefect.client and secrets. i also want help with connecting to my prefect cloud

try to use prefect on a local kubernetes cluster where to mounts prefect flows in kubernetes cluster and execute script inside cluster without creating any docker image for each of my flows

Hello All, I'm new to prefect and can someone help me if we can connect prefect to Azure Keyvault and use the secrets in the prefect flows? I couldnt find any Docs online. thanks in advance.

how to set work queue limit in prefect with CLI? work queue NOT work-pool

Prefect Interval Schedules: If the prefect agent is not running for a while... does it try to pick up missed deployment intervals, or does the interval clock start over on agent restart? Is interval trigger managed by agent or orion db server?

Is there a way to ensure a deployment is only running one at a time? ie if deployment X is running, another run should not start until it completes.

Scheduled Jobs Piles Up: Hi guys I’m just wondering what would happen if say I schedule a job to run every 30 mins but each job takes about 45mins, is there a way to easily squash all the later queued jobs so it doesn’t pill up? Thank you!

Hi all, Just getting started with Prefect, with some limited exposure to Airflow. Prefect looks promising but I could use your input for the deployment question below. To keep the cost low, I want to run my orchestration tool with either of the two options: a) self managed using AWS EC2 b) managed EKS on AWS Airflow can be deployed in both these ways. Can I deploy Prefect 2.0 (open source) using both/any one of these options? I read the docs(https://docs.prefect.io/concepts/deployments/) and some blogs on how people deployed Prefect but I am still confused. Again, I want to use the open source version of Prefect 2.0 and deploy it using Kubernetes on self managed AWS EC2 or AWS EKS. Thanks for your feedback.

Hi I'm trying to use a bitbucket storage block with a private repo. I've setup a bitbucket credentials block which is authenticated like this, using username and password. Then I use that inside a bitbucket repository block. But I get this error:

token = self.bitbucket_credentials.token.get_secret_value() AttributeError: 'NoneType' object has no attribute 'get_secret_value'

does anyone have any ideas? The blocks UI does suggest personal access token is optional. Unless it's 'optional' in case it's a public repo

[Deleted message because I rubberducked myself lol]

Hello everyone, I noticed that Prefect downloads my code when running a flow. How can I prevent it from downloading my code? My code contains many data files, which result in large disk space occupation.

My code and Prefect are deployed on the same machine, but when starting the Prefect server, I used the command: ‘prefect server start --host=0.0.0.0’

thanks for your reply, but how can i know the “BLOCK_NAME” for the specific path?

When I was importing blocks, a new error occurred. Do I need to manually add this ‘.prefectignore’?

My code is located at /app/wk/EGES. I first set up a Block and added /app/wk/EGES to the eges BLOCK. Then, when defining the flow, I loaded the eges BLOCK. However, when running the flow, the code is still being downloaded from /app/wk/EGES.

Hi everyone, I am a new user to Prefect and I was able to get the setup to work locally with everything just on my computer. But now I am trying to deploy it to the cloud set up. I have my Prefect cloud workspace, but now I need to have the agent set up and the storage block for the code. I plan for the storage block to exist in Azure. I met with the DevOps cloud engineer at my company, and he was able to create a service principal for the container registry resource I plan to use. But now I need to figure out how to configure a storage block and deploy the code to it. Can anyone assist with this? I think after this I can try and run the agent even locally to run the scheduled tasks. I only need to test this for a simple example and then I think I can take it from there once I know that it works. Another big part of this is being able to deploy these with a docker image, which I am not 100% familiar on how to do.

I looked up the documentation for build_from_flow at docs, but it didn’t mention the file path parameter. How can I learn about how to use this parameter?

build_from_flow can take all the same parameters as the

Deployment

class, as demonstrated here: https://docs.prefect.io/api-ref/prefect/deployments/?h=build_from_flow#prefect.deployments.Deployment

This message was deleted.

Are there any known issues with

pyodbc

? I keep running into

pyodbc.ProgrammingError: No results.  Previous SQL was not a query.

I'm getting a crash, which ends up hanging the terminal. On my Windows laptop I must close the terminal as

Ctrl+C

does nothing (on Linux I get an even longer trace). I have been able to reproduce with a minimal ~30 line example. Here is the example code, it's output, as well as the longer trace from my full program: https://gist.github.com/ZachMassia/efe79e6853afdf20b367ff5bbd2825f3 Thanks in advance if anyone can point me in the right direction!

I'm looking for more information on some basics on running a dockerized script that communicates with prefect. I currently have a docker-compose yaml script that takes a postgredb and then I execute a dockerized script that uses sqlalchemy block. My prefect server is on the host computer. (I'll try dockerizing prefect later). Here is the command:

docker run -t   --network=bc-dev_default -v /home/j/project/sales:/src sales_ingest:v002  --filename=/src/sales.csv

Here is the tail of the error:

ValueError: Unable to find block document named postgres-dev for block type sqlalchemy-connector

The name matches the block named shown in the UI. Here is my Dockerfile

FROM python:3.9

RUN apt-get install wget

WORKDIR /app

COPY requirements.txt .
COPY ingest_sales_flow.py ingest_sales_flow.py

RUN pip install -r requirements.txt

ARG PREFECT_API_URL
ENV PREFECT_API_URL=$PREFECT_API_URL

ENTRYPOINT [ "python", "ingest_sales_flow.py" ]

To access a server running on the host machine from inside a Docker container, you might need to set PREFECT_API_URL to

<http://host.docker.internal:4200>

. If you're running Docker on Linux (and not Desktop Desktop on Windows or macOS) you'll also need to add

--add-host=host.docker.internal:host-gateway

to your

docker run

command.

Ah, thank you @Ryan Peden. I'm experimenting with another script right now. I will try that out today.

These did not work for me.

I finally figured it out. However, I went with the dockerized prefect route. Your Gandalf post helped out along the way as well 😂.

hello all.....can anyone tell me with Python 3.8, 3.9 Installed in my VM......can anyone confirm if prefect 2 works on Ubuntu 18.04 LTS

When running prefect flows locally, flows occasionally crash with

MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.

Rerunning the flow generally seems to take care of it. Has anyone else seen this / dug into what is causing it? This happens frequently with long running flows.

How many tasks/flows can one agent run simultaneously?

Hello, I'm migrating from prefect 1 to prefect 2. Is there an equivalent to DockerStorage in Prefect 2 blocks ? I found the docker registry block but its not a storage block if I understood well. I'm currently storing my flows in dockerhub, each one needing a different custom image, building directly from prefect parameters was easy I'd rather avoid creating those images separately 🙂