https://prefect.io
Join Slack
Hi guys. I was not able to find any solution for my current problem. Is it possible to delete flow r...
m

Martin Durkac

almost 4 years ago
Hi guys. I was not able to find any solution for my current problem. Is it possible to delete flow run history from specific day like yesterday or in highly repeated flows delete history older than 5 hours? I ran into problem with my memory on EC2, because my flow generated 11000 flow runs and EC2 instance run out of memory. I need to make some script which removes older runs based on flow id or name. Thanks for your help and ideas. 🙂
m
a
+2
  • 4
  • 8
  • 103
Hi, can someone explain how exactly heartbeats work? I have a flow that reads a ton of data from sno...
j

Justin Liu

about 4 years ago
Hi, can someone explain how exactly heartbeats work? I have a flow that reads a ton of data from snowflake and writes them to a series of tsv files, and when the run is about 20 minutes in it gets marked as failed by the Zombie killer, even though I know it’s still running. It may be helpful to mention that it’s running through a nested for loop. But digging a little deeper, I noticed that the code was printing logs about once every minute, and since the Zombie killer waits for no signal after 2 minutes I find it odd that it acts. Also, the code worked successfully when heartbeats were turned off, I’m just wondering if there is a better alternative in case this happens in the future.
j
k
z
  • 3
  • 9
  • 103
<@ULVA73B9P> Can you show me an example where you deploy multiple flows at once programmatically?
j

Jackson Nguyen

8 months ago
@Marvin Can you show me an example where you deploy multiple flows at once programmatically?
✅ 1
j
m
  • 2
  • 21
  • 102
<@ULVA73B9P> when using a custom image for a prefect kubernetes workpool can i use a private registr...
d

Dominik

12 months ago
@Marvin when using a custom image for a prefect kubernetes workpool can i use a private registry?
d
m
  • 2
  • 10
  • 102
Does prefect have a good way to scale horizontally in Prefect Server?
c

Charles Leung

over 1 year ago
Does prefect have a good way to scale horizontally in Prefect Server?
c
j
  • 2
  • 3
  • 102
Hello! I am attempting to override the default memory/cpu limit/requests for deployments that are cr...
d

Dane Linsky

over 1 year ago
Hello! I am attempting to override the default memory/cpu limit/requests for deployments that are created by a flow. Today we are using the infra_overrides args in order to get our image to populate when the K8s worker spins up pods, and that seems to work just fine. But, it does not appear to be changing the resource allocation out of the default values. The deployment config created through build_from_flow shows up in the as the photo below. Is infra_overrides the correct method of setting these on a per deployment basis or is there another value we need to be setting in order to manage resources more effectively in the pods spun up?
👀 0
✅ 1
d
a
m
  • 3
  • 5
  • 102
What are our options for external monitoring or webhook integrations for total system uptime for Pre...
c

Charles Liu

almost 4 years ago
What are our options for external monitoring or webhook integrations for total system uptime for Prefect? Looking for the ability to alert for outages.
c
k
  • 2
  • 8
  • 102
Hi everyone, Wondered if exponential backoff on retries was some that Prefect might have on a roadma...
j

John Shearer

almost 4 years ago
Hi everyone, Wondered if exponential backoff on retries was some that Prefect might have on a roadmap, or people have interest in? Airflow has this feature (retry_exponential_backoff). My use cases are APIs with rate limits, and backing off automatically when we hit them
j
a
z
  • 3
  • 3
  • 102
<@ULVA73B9P> how can I trigger a flow run from receiving a SQS message?
f

Fabien Punin

8 months ago
@Marvin how can I trigger a flow run from receiving a SQS message?
f
m
  • 2
  • 3
  • 101
Hello, Is there a way to set the cache_policy to None by default? Most of our task needed to be reru...
j

Jezreel Zamora

10 months ago
Hello, Is there a way to set the cache_policy to None by default? Most of our task needed to be rerun and the caching is causing multiple issues on our tasks.
j
b
  • 2
  • 5
  • 101
Previous596061Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by