https://prefect.io
Join Slack
I am trying to transition our deployments from a Cloud Run worker pool to a Cloud Run V2 worker pool...
i

Isaac

over 1 year ago
I am trying to transition our deployments from a Cloud Run worker pool to a Cloud Run V2 worker pool in order to leverage the longer timeouts >3600s. I have been doing testing but have had issues getting it to work. Any time I run a deployment on the worker pool I get the following error:
Flow run could not be submitted to infrastructure: <https://run.googleapis.com/v2/projects/trading-system-dev/locations/us-central1/jobs?jobId=prefect-cyber-sparrow&alt=json> returned "Violation in CreateJobRequest.job.template.template.vpc_access.connector: one and only one of connector and network_interfaces must be set.". Details: "[{'@type': '<http://type.googleapis.com/google.rpc.BadRequest|type.googleapis.com/google.rpc.BadRequest>', 'fieldViolations': [{'field': 'Violation in CreateJobRequest.job.template.template.vpc_access.connector', 'description': 'one and only one of connector and network_interfaces must be set.'}]}]">
From what i've gathered, prefect uses Terraform and cloud run seems to require either a connector or network_interfaces. I am not specifying a vpc connector (which prefect has marked as optional). It almost seems as though when there is no vpc connector, the network_interfaces should be used as a default but is not happening. Any help or advice would be greatly appreciated.
i
j
n
  • 3
  • 17
  • 134
If I have a StartFlowRun task with wait=True in my flow, and the subflow fails, what is the expected...
m

Matthew Seligson

over 3 years ago
If I have a StartFlowRun task with wait=True in my flow, and the subflow fails, what is the expected behavior when I restart the parent flow? Should the StartFlowRun schedule a brand new subflow run? Should the subflow restart all failed tasks? Is this documented anywhere?
m
a
k
  • 3
  • 32
  • 134
My question is regarding Storage. How does anything change if I add `flow.storage=Local()` to my He...
t

Tilak Maddy

almost 4 years ago
My question is regarding Storage. How does anything change if I add
flow.storage=Local()
to my Hello world flow ? Everything seems to be the same
t
a
  • 2
  • 27
  • 134
<@ULVA73B9P> in Prefect 3, how do I restart a failed job? What are the requirements to allow restart...
v

Victoria

10 months ago
@Marvin in Prefect 3, how do I restart a failed job? What are the requirements to allow restarting?
v
m
  • 2
  • 5
  • 133
<@ULVA73B9P> how to get the return value of the flow using flow id?
j

Joish

over 1 year ago
@Marvin how to get the return value of the flow using flow id?
j
m
+2
  • 4
  • 20
  • 133
<@ULVA73B9P> how can i debug the prefect code in my local machine
s

Suman Kamalapuram

over 1 year ago
@Marvin how can i debug the prefect code in my local machine
s
m
  • 2
  • 1
  • 133
Has anybody migrated from 1.x to Prefect 2? And how was the journey?
a

Andrei Tulbure

almost 3 years ago
Has anybody migrated from 1.x to Prefect 2? And how was the journey?
marvin duck 2
a
k
+2
  • 4
  • 5
  • 133
Hi, We are exploring alternative options to Airflow and were wondering if Prefect is compatible with...
s

Shriram Holla

over 3 years ago
Hi, We are exploring alternative options to Airflow and were wondering if Prefect is compatible with build systems. We use bazel and some of our scripts are only visible when run through bazel. Can we configure flows to run through a build system?
s
a
  • 2
  • 9
  • 133
I'm trying to get a DaskExecutor on ECS/Fargate to work. And having issues. The flow hangs on `Creat...
m

Marc Lipoff

over 3 years ago
I'm trying to get a DaskExecutor on ECS/Fargate to work. And having issues. The flow hangs on
Creating a new Dask cluster with None.fargate_cluster...
. I'm running on prefect==0.15.6. My code is attached. The trouble is that it just hangs and I have no clue what it's "trying" to do.
m
k
  • 2
  • 25
  • 133
<@ULVA73B9P> is there a way to explicitly clear memory in prefect for large-running flows?
k

Kevin McCraney

over 1 year ago
@Marvin is there a way to explicitly clear memory in prefect for large-running flows?
k
m
  • 2
  • 7
  • 132
Previous424344Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by