https://prefect.io
Join Slack
the machine I was using to host the prefect server failed over the weekend. When i restarted the mac...
j

joshua mclellan

over 3 years ago
the machine I was using to host the prefect server failed over the weekend. When i restarted the machine and tried running
prefect server start --expose --use-volume
its not using the existing data/configurations i set up and in the logs im seeing the following messages:
hasura_1    | {"type":"startup","timestamp":"2022-03-21T19:50:54.856+0000","level":"error","detail":{"kind":"catalog_migrate","info":{"path":"$","error":"Cannot use database pr
eviously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47).","code":"not-supported"}}}
hasura_1    | {"path":"$","error":"Cannot use database previously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47)."
,"code":"not-supported"}
how do I go about debugging this?
j
k
a
  • 3
  • 4
  • 94
Hi guys. I am running prefect server locally and trying to access it using the ip_address which i al...
f

Faizan Qazi

almost 4 years ago
Hi guys. I am running prefect server locally and trying to access it using the ip_address which i also configured in the
config.toml
file . But as i access it at
<http://ip.add.re.ss:8080>
, the page displays but then redirects to the getting started page . So basically i want to access my prefect instance from any other machine.
[server]
  [server.ui]
  apollo_url="<http://ip.add.re.ss:4200/graphql>"
f
a
  • 2
  • 15
  • 94
<@ULVA73B9P> is it possible to create a work pool from prefect.yaml if so how ?
d

datamongus

6 months ago
@Marvin is it possible to create a work pool from prefect.yaml if so how ?
d
m
  • 2
  • 1
  • 93
<@ULVA73B9P> I want to give my task run a specific name. Can I assign a variable inside my task and ...
k

Koen

7 months ago
@Marvin I want to give my task run a specific name. Can I assign a variable inside my task and use that as the task run name? Or must the task run name always be passed on in the task parameters?
k
m
  • 2
  • 1
  • 93
<@ULVA73B9P> I'm using Prefect 3 and trying to deploy my first flow on an ECS fargate push work pool...
f

Fabien Punin

9 months ago
@Marvin I'm using Prefect 3 and trying to deploy my first flow on an ECS fargate push work pool created using the prefect cli. I'm baking the flow and task code directly into a docker image stored on a private ECR registry. Below is my prefect.yaml file where the image name and tag are properly populated (it is stored in ecr).
name: test
prefect-version: 3.1.6

deployments:
- name: test_flow_x_hourly_deployment
  version: "1"
  tags: null
  description: This deployment runs flow x hourly
  schedules:
    - cron: "0 0 * * *"
      timezone: "Europe/Paris"
      active: "true"
  flow_name: flow_1
  entrypoint: src/projects/test/prefect/flows/flow_1.py:sample_flow
  parameters:
    input_data: "Hello, World!"
  work_pool:
    name: ecs-fargate-workpool
    work_queue_name: default
    job_variables:
      image: '{{ $IMAGE_NAME }}:{{ $IMAGE_TAG }}'
Below is my dockerfile from which the image is built
FROM prefecthq/prefect:3.1.6-python3.11

ARG PROJECT_PATH

WORKDIR /app

RUN pip install uv

COPY uv.lock .
COPY pyproject.toml .

RUN uv sync

COPY src/common src/common
COPY ${PROJECT_PATH} ${PROJECT_PATH}

ENV PATH=".venv/bin:$PATH"

ENTRYPOINT []
My flow crashes with the error
Flow run infrastructure exited with non-zero status code:
 Exited with non 0 code. (Error Code: 1)
This may be caused by attempting to run an image with a misspecified platform or architecture.
What could I be missing?
f
m
+2
  • 4
  • 32
  • 93
<@ULVA73B9P> I'm trying to the Prefect over my pipeline which involves usage of a database component...
s

Sachin Mandhotra

10 months ago
@Marvin I'm trying to the Prefect over my pipeline which involves usage of a database component in various tasks. During the flow run, I keep getting below error warnings -
raise HashError(msg)
prefect.exceptions.HashError: Unable to create hash - objects could not be serialized.
  JSON error: Unable to serialize unknown type: <class 'database.PostgreSQLDB'>
  Pickle error: cannot pickle '_contextvars.Context' object
Can you help in resolving the root cause here?
s
m
  • 2
  • 4
  • 93
<@ULVA73B9P> I have a fastapi service where I have implemented a lot of the functionality that I nee...
b

Benjamin Rapaport

10 months ago
@Marvin I have a fastapi service where I have implemented a lot of the functionality that I need for running prefect workflows. Within this service, I rely on the fastapi dependency injection mechanism to construct and inject my dependencies into routes and various classes. How would you suggest migrating to prefect?
b
m
  • 2
  • 1
  • 93
<@ULVA73B9P> explain this error and what can cause it ```ERROR | prefect.infrastructure.kubernetes...
d

Dominick Olivito

11 months ago
@Marvin explain this error and what can cause it
ERROR   | prefect.infrastructure.kubernetes-job - Job 'prefect-job-qqjvt': Job reached backoff limit.
d
m
  • 2
  • 13
  • 93
<@ULVA73B9P> I am trying to create a prefect.yaml file using prefect version 2.x I get this error wh...
m

Maja

about 1 year ago
@Marvin I am trying to create a prefect.yaml file using prefect version 2.x I get this error when I try to use the command prefect deploy in the terminal: You've chosen a work pool with type 'prefect-agent' which cannot be used for project-style deployments. Let's pick another work pool to deploy to. My deployment section of the prefect.yaml file looks like this: deployments: - name: introduction.template.test_flow version: null tags: [] description: null schedule: {} flow_name: null entrypoint: template\test_flow\flow.py:test_new_deploy_flow parameters: {} work_pool: name: default-agent-pool work_queue_name: prod-limit-1 job_variables: image: '{{ build_image.image }}' How do I create a prefect.yaml file that can deploy to a work pool with type prefect-agent?
m
m
  • 2
  • 9
  • 93
Hi folks, I'm new here. I am looking to use Prefect as an orchestrator and primary development focus...
t

Tyler Matteson

over 3 years ago
Hi folks, I'm new here. I am looking to use Prefect as an orchestrator and primary development focus, but I also am interested in leveraging work that has already been done in Singer.io and the Meltano runner specifically. Choosing Meltano because my team has a background in Python and Vue and not in Airbyte's stack of Java and React. First level of understanding is how I might/should pass and retrieve data between Prefect and Meltano without a direct integration (perhaps with a call to
subprocess
). What should I know? What am I not asking? Tasks are a mix of polling, ETL on demand and scheduled ETL tasks. I think most data professionals would describe the load as "not much", so efficiency is going to take back seat to maintainable and easy to use.
t
a
  • 2
  • 21
  • 93
Previous666768Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by