https://prefect.io
Join Slack
Hello everybody! I've been playing around with the idea of self hosting Prefect server on a VM inst...
a

Aaron Gonzalez

over 2 years ago
Hello everybody! I've been playing around with the idea of self hosting Prefect server on a VM instance using docker compose. The following posts have gotten me pretty much all the way to having a working self-host option: • https://dev.to/calvinqc/the-easiest-docker-docker-compose-setup-on-compute-engine-1op1 • https://medium.com/the-prefect-blog/how-to-self-host-prefect-orion-with-postgres-using-docker-compose-631c41ab8a9f I am able to just set my local
PREFECT_API_URL
env var to the external IP address of my vm instance (not the nicest solution but I can clean this up later) and then I can immediately push my blocks and deployments to the server! 👍 The problem comes from when I lauch a run of one of my test deployments that use a Cloud Run Job. I never get out of the "Pending" status in the Prefect UI. Pretty sure it is something with my networking and probably not an actual issue with Prefect, but I can't seem to make any progress. I've tried setting up Serverless VPC Connections to allow the Cloud Run Job to be able to connect to the vm that has the server running, but maybe I've set it up incorrectly 🤷. Anyone else ever managed to get a cloud run job deployment to work with a vm instance hosted prefect server?
a
n
r
  • 3
  • 4
  • 192
Hi, I'm new to Prefect. It's fantastic to deploy pipelines by using Python code. However, I had face...
h

Hash Lin

over 3 years ago
Hi, I'm new to Prefect. It's fantastic to deploy pipelines by using Python code. However, I had faced some issues when deploying a pipeline to Prefect. Here's my scenario. • I want to run Prefect Agent in the K8s cluster, and I successfully plug the agent into our cluster. • I built a docker image in Gitlab CI and pushed the docker image to ECR. Then, I wrote the Dockerfile and reference from here. • I write a flow script and register to the Prefect cloud by using my local computer. Unfortunately, after running the pipeline, I got some errors: the python model was not found. As a result, I can't successfully import the module that I wrote in the project. And here's my flow script.
Failed to load and execute flow run: ModuleNotFoundError("No module named '/Users/xxx/'")
Thanks for helping. 🙇
h
k
+2
  • 4
  • 29
  • 192
I'm nearly there on getting my first successful flow run with an ECS work pool, but I keep getting a...
n

Nash Taylor

about 1 year ago
I'm nearly there on getting my first successful flow run with an ECS work pool, but I keep getting a Crashed run with this error:
Flow run infrastructure exited with non-zero status code:
 Exited with non 0 code. (Error Code: 1)
This may be caused by attempting to run an image with a misspecified platform or architecture.
I've been following this guide (https://docs.prefect.io/3.0/deploy/infrastructure-examples/serverless) and the
prefect work-pool create
step went fine. What am I missing here?
n
b
  • 2
  • 3
  • 190
<@ULVA73B9P> Can I run Docker with GPUs on Prefect?
p

Petko

over 1 year ago
@Marvin Can I run Docker with GPUs on Prefect?
p
m
  • 2
  • 5
  • 190
<@ULVA73B9P> is there a way to get my flow state using flow id?
j

Joish

about 2 years ago
@Marvin is there a way to get my flow state using flow id?
j
m
n
  • 3
  • 3
  • 190
Does anyone have know how to integrate Loguru with Prefect? I don't want to use `log_prints=True`, b...
m

Morten Hauge

10 months ago
Does anyone have know how to integrate Loguru with Prefect? I don't want to use
log_prints=True
, because then all messages will be logged as
INFO
. I want to use Loguru instead of
get_run_logger
because my code is calling out to internal libraries that log messages with Loguru and provide contextual information using
.bind
, and I need these messages to appear as well in the Prefect UI.
m
b
  • 2
  • 9
  • 189
<@ULVA73B9P> failed to authorize: failed to fetch anonymous token:
p

Parash

about 1 year ago
@Marvin failed to authorize: failed to fetch anonymous token:
p
m
  • 2
  • 1
  • 189
Hi! I want to debug a flow-of-flows on my local machine, but the flow runs created using create_flow...
j

Jonathan Mathews

over 3 years ago
Hi! I want to debug a flow-of-flows on my local machine, but the flow runs created using create_flow_run seem to be executing on my production Prefect Cloud environment. How do I ensure that these flow runs are all executed on my local machine?
j
k
c
  • 3
  • 6
  • 189
<@ULVA73B9P> I don’t see my deployments or the flows on the dashboard, but I can see them when I go ...
a

Ajeel Ahmed

about 1 year ago
@Marvin I don’t see my deployments or the flows on the dashboard, but I can see them when I go to the cli and run
prefect deployment ls
what could be the problem?
a
m
  • 2
  • 12
  • 188
<@ULVA73B9P>, if I run `prefect worker start` instead of `prefect agent start`, I get this error ```...
g

Giacomo Chiarella

about 1 year ago
@Marvin, if I run
prefect worker start
instead of
prefect agent start
, I get this error
Discovered type 'prefect-agent' for work pool '<>'.
'prefect-agent' typed work pools work with Prefect Agents instead of Workers.
Please use the 'prefect agent start' to start a Prefect Agent.
can you tell me how to solve it?
g
m
  • 2
  • 38
  • 188
Previous242526Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by