https://prefect.io
Join Slack
All, I want to delete 200+ flow run that are in pending state, I can filter them in UI. However, I ...
p

Patrick Tan

almost 3 years ago
All, I want to delete 200+ flow run that are in pending state, I can filter them in UI. However, I am not able to 'select all' and delete all at once. I have to select one check box at a time. Any short cut ?
👀 1
p
k
+3
  • 5
  • 4
  • 73
Hi :slightly_smiling_face: Does prefect 2.0 export some metrics through an open-telemetry exporter f...
l

Lucien Fregosi

about 3 years ago
Hi 🙂 Does prefect 2.0 export some metrics through an open-telemetry exporter for example ? 🙂 And more globally what is the best way to be alerted when a job fails ? (I have already an alert in my kubernetes cluster if a pod fails but maybe it’s possible to have something more prefect related/high level)
✅ 1
l
s
g
  • 3
  • 4
  • 73
I observed that the pods start and keep failing and restarting While no resources show up when we ge...
k

Kayvan Shah

over 3 years ago
I observed that the pods start and keep failing and restarting While no resources show up when we get pods for default namespace
error.txt.py
k
a
  • 2
  • 30
  • 73
Hi everyone, This may seem a bit strange but I’m planning to use a Google Sheet as a front end where...
k

Ken Nguyen

over 3 years ago
Hi everyone, This may seem a bit strange but I’m planning to use a Google Sheet as a front end where users can type in inputs. Then, users can click the submit button to trigger a Prefect flow that will read the Gsheet’s inputs, transform it, then write it back onto the Gsheet. I’m wondering if anyone has tried something similar and can offer feedback on my approach: 1. We have a button in Gsheets that will activate a Google Apps Script to do an API call 2. The API call (hosted on AWS API Gateway) will trigger a Lambda function, which will trigger a Prefect flow 3. Prefect flow will read data in Gsheets, transform the data, then write data in Gsheets Any suggestions or input is appreciated!
k
a
  • 2
  • 6
  • 73
In one of my flows, I have a task to upload a video to s3 (boto3 s3 resource `upload_file()` call, b...
m

Manuel Gomes

almost 4 years ago
In one of my flows, I have a task to upload a video to s3 (boto3 s3 resource
upload_file()
call, because big&binary). Since it's synchronous, I can trust the file will be where desired when task succeeds. Next task in this flow is to transcode this video. I do so by creating a
mediaconvert
boto3
client, and sending a mess of json to its
create_job(**args)
method. This returns me... the job. Now from what I've read... I should be able to use a prefect built-in
prefect.tasks.aws.client_waiter.AWSClientWait
to wait for said job to finish (which is fine, at this point the workflow is serial/synchronous). Problem is... even when the job reports success (in the console, even!), it takes a while (minutes?!) for the transcoded movie to be present in the target bucket. I would then... need to enter another wait task until I could find the file in the bucket's list of objects, possibly through prefect.tasks.aws.s3.S3List? until I could proceed to do further things to this transcoded video? This conjunction sounds all too common not to have an integrated solution, unless I'm being dense (hah! no news there!) and not spotting an obvious solution. Any guidance?
m
a
  • 2
  • 5
  • 73
Hey all! I am working on to integrate splunk with prefect agent. I need to know where all the LOGS b...
s

serhan

almost 4 years ago
Hey all! I am working on to integrate splunk with prefect agent. I need to know where all the LOGS being written, what is the path for the log files? Can anyone help?
s
k
a
  • 3
  • 2
  • 73
<@ULVA73B9P> how do you specify pool size when creating a sqlalchemy connection block?
j

Josh Dunkley

6 months ago
@Marvin how do you specify pool size when creating a sqlalchemy connection block?
j
m
  • 2
  • 7
  • 72
Hello: I am seeing a message Work queue does not have any actively polling workers ready to execut...
c

Cormac

7 months ago
Hello: I am seeing a message > Work queue does not have any actively polling workers ready to execute work. But can find no documentation on this. Can someone assist?
c
  • 1
  • 1
  • 72
<@ULVA73B9P>, how can I define a task with a specific timeout and retries?
p

Pascal Jahan Elahi (Pawsey)

8 months ago
@Marvin, how can I define a task with a specific timeout and retries?
p
m
  • 2
  • 11
  • 72
hey guys, is there a way to use .map() and .submit() in asyncronous tasks in prefect 3? i'm trying t...
g

Guilins

9 months ago
hey guys, is there a way to use .map() and .submit() in asyncronous tasks in prefect 3? i'm trying to upgrade from version 2 but all my flows are async and anytime i try to await a mapped task i get this error: TypeError: object PrefectFutureList can't be used in 'await' expression
g
j
  • 2
  • 2
  • 72
Previous888990Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by