Hi. Are (some) values from config.toml exposed through the GQL API? I want to confirm the current active GQL query timeout value, but don't have direct access to the server to check the config.toml file.

Hey there again everyone! Hopefully a quick question about doing a

create_flow_run.run(flow_name="flow-2")

within a task created by a

KubernetesRun

... assuming i have

with Flow("flow-1", run_config=KubernetesRun(...)):
  ... creates task that does a create_flow_run.run(flow_name="flow-2") ...

should

flow-2

receive/inherit the

run_config

from

flow-1

?

How are others handling the occasional ECS/Fargate error

Timeout waiting for network interface provisioning to complete.

?

Hi I am using prefect to orchestrate my spark jobs. The spark jobs are submitted with

spark-submit

using prefect’s

ShellTask

. I have created a subclass of ShellTask to invoke the `spark-submit.`The spark jobs are running on k8s. I am facing an issue where prefect tasks are not completing and continuously running. I see this happening on the tasks which are slightly long running (> 10 mins). The master flow maps over list and orchestrates the prefect

ShellTask

e.g.

K8sSparkSubmitTask.map(
  id = ["id1", "id2", "idx"]
)

The task starts running for

id1

and pod status is observed to be

successful

. However prefect does not move to the next task. Why are these tasks not getting completed?

Hi! Running this:

helm pull --destination /tmp/c68c98cd-c919-46b6-9e17-924f22aa2dd3 --version 2022.01.25 --repo <https://prefecthq.github.io/server> prefecthq/prefect-server

results in an error :

Error: chart "prefecthq/prefect-server" version "2022.01.25" not found in <https://prefecthq.github.io/server> repository

However, I do see

2022.01.25

as a version when I do

helm search repo prefecthq/prefect-server --versions

. Can someone please point out where I am going wrong?

Hey, I was wondering if there is a mechanism to send a notification when a flow run has been executing for a certain length of time. We have a state handler that notifies us to failed tasks, but a long running flow may not necessarily change states, so I’m not sure if there is another mechanism we can use beyond state handlers.

Python 3.10 despite the tuple typing annotation there is still

TypeError: Task is not iterable

exception.

@task(timeout=timedelta(seconds=30))
def fetch_symbols(...) -> tuple[MyCustomType, ...]:
  ...

I’m wondering if it’s possible for Prefect to run the same Flow at the same time with different Parameters? Let’s say i have a flow call Balance and i want to run two or more instances at the same time but with different input Parameters, is that possible?

I upgraded to My Orion instance last night to 2.0beta with the server running. It looked like everything worked from a UI perspective. However, when I run a flow they all fail with "Crash detected! Execution was interrupted by an unexpected exception. I am thinking I should probably stop all of the Prefect process and restart the Server. I don't see a command line or an option in the UI to stop or restart the Orion Server.

is there a way to validate flows before registering them? i’d like to test things locally like storage, etc. that would otherwise cause an error on the registering step

My prefect agent stopped working and is showing: `prefect.exceptions.ClientError: Your Prefect Server instance has no tenants. Create a tenant with `prefect server create-tenant`` When I try to run

prefect server create-tenant --name default --slug default

I'm getting the following error:

prefect.exceptions.ClientError: [{'message': 'Uniqueness violation.', 'locations': [{'line': 2, 'column': 5}], 'path': ['create_tenant'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Uniqueness violation.'}}}]

It seems that the tenant already exists. Do you know what might be the problem?

Hi everyone, I'd like to try Orion, I noticed that there is some asyncio support using AnyIO. However, I don't see anything listed in the task runners documentation. Is this feature disabled for now?

One more question, is there a docker-compose file somewhere for Orion to get started quickly? I'd rather this than using a cli. With the previous version, I had to reverse engineer the cli to create one.

Is there an architectural diagram available that showcases prefect with an aws infrastructure?

I’m attempting to grab the logger out of the prefect context. I am doing something similar to this

@task()
def my_task():
    import prefect
    logger = prefect.context.get('logger')

I’ve tried it both with having the

import prefect

at the top level outside of the task and in the task like in that example and both times it gives me this

TypeError: Pickling context objects is explicitly not supported. You should always access context as an attribute of the `prefect` module, as in `prefect.context`

Anyone have any ideas why it’s failing when i try to register the flow?

tl;dr How scalable is Prefect Server for scheduling concurrent runs of 10s of thousands of flows? I'm evaluating Prefect Server as a solution for managing a data pipeline. How I envisioned this working was using the Prefect framework as a computationally minimal "glue." Prefect would be the central pipe through which the state of the data flows (but not the data itself). It would hand off all computation to microservices via (a)sync requests and its only concern would be controlling how the state is pushed from service to service. Prefect seems to be a great tool for this job; I really like the API and how quickly a developer can get up and running. However, there are a couple of things that I don't yet have total clarity on. This documentation describes Server as a "highly scalable scheduler" but also says that it may start degrading at "~10-20 tasks running concurrently against a typical Server." Does the above degradation occur because the

typical Server

is deployed to a single node via docker-compose with a single agent, instead of being deployed via something like Helm, which horizontally scales scheduled flows across a cluster of agents? How quickly does the size of the PostgreSQL database generally grow, relative to the number of flows run? Is there any reason that regularly deleting old runs would be any more complicated than as suggested here? Are there any case studies with Prefect Server being used to run 10s of thousands of concurrent flows? I see that there used to be a Nomad Agent, are there any helpful resources for running Prefect on a nomad cluster? Thanks for any advice you are able to share. I have been diving into Prefect for only the past couple of days so greatly appreciate any referential pointers here.

Hi all! I'm having issues with my Hasura pods. It just keeps restarting every once in a while and I'm not sure about why this is happening. The values for the helm chart I'm using for Hasura are the following:

hasura:
  image:
    name: hasura/graphql-engine
    tag: v1.3.3
    pullPolicy: IfNotPresent
    pullSecrets: []

  service:
    type: ClusterIP
    port: 3000

  labels: {}
  annotations: {}
  replicas: 2
  strategy: {}
  podSecurityContext: {}
  securityContext: {}
  env: []
  resources:
    limits:
      cpu: "500m"
      memory: "1Gi"
    requests:
      cpu: "100m"
      memory: "256Mi"
  nodeSelector: {}
  tolerations: []
  affinity: {}

both the pods crash at the same time and keep restarting a few times before it's back up, so my Server gets a significant amount of downtime. I don't think the resources are underestimated, but I do see a pattern on RAM usage when this happens (I've sent a screenshot). I've also attached some logs for the pods, but I can't find any relevant information on them. Is there anywhere else I could gather information for this issue?

i have prefect-server (including apollo, graphql, hasura, postgres, towel and ui) and a docker agent running in a local docker-compose environment and i am able to kick off a flow run with the cli

prefect run --name ...

but the agent doesn't seem to actually run the flow logic. Watching the prefect agent logs i see:

agent_1        | [2022-03-21 05:41:42,906] INFO - agent | Registering agent...
agent_1        | [2022-03-21 05:41:42,926] INFO - agent | Registration successful!
agent_1        | 
agent_1        |  ____            __           _        _                    _
agent_1        | |  _ \ _ __ ___ / _| ___  ___| |_     / \   __ _  ___ _ __ | |_
agent_1        | | |_) | '__/ _ \ |_ / _ \/ __| __|   / _ \ / _` |/ _ \ '_ \| __|
agent_1        | |  __/| | |  __/  _|  __/ (__| |_   / ___ \ (_| |  __/ | | | |_
agent_1        | |_|   |_|  \___|_|  \___|\___|\__| /_/   \_\__, |\___|_| |_|\__|
agent_1        |                                            |___/
agent_1        | 
agent_1        | [2022-03-21 05:41:42,933] INFO - agent | Starting DockerAgent with labels ['default']
agent_1        | [2022-03-21 05:41:42,933] INFO - agent | Agent documentation can be found at <https://docs.prefect.io/orchestration/>
agent_1        | [2022-03-21 05:41:42,933] INFO - agent | Waiting for flow runs...
agent_1        | [2022-03-21 05:44:39,049] INFO - agent | Deploying flow run 71f0c5d3-b73b-4537-8267-b0fd30325fd3 to execution environment...
agent_1        | [2022-03-21 05:44:39,082] INFO - agent | Pulling image localhost:5000/my-flow:2022-03-21t05-43-58-036580-00-00...
agent_1        | [2022-03-21 05:44:39,284] INFO - agent | Successfully pulled image localhost:5000/my-flow:2022-03-21t05-43-58-036580-00-00
agent_1        | [2022-03-21 05:44:39,689] INFO - agent | Completed deployment of flow run 71f0c5d3-b73b-4537-8267-b0fd30325fd3

Is it not actually running the flow logic or do the logs not show up on the agent?

Hey all, we recently updated our prefect cluster from version 0.15.10 to major version 1.0.0. We used an existing DaskCluster to distribute our flowruns, which we could simply do by setting envs

PREFECT__EXECUTOR__DEFAULT_CLASS

and

PREFECT__EXECUTOR__DASK__ADDRESS

in our agent. Unfortunately, this approach doesn't seem to work anymore. I'm pretty sure we could initialize a DaskExecutor in our code and attach it to our flow somehow, but this would be undisireable, as it requires changing each of our existing flows, but also for each flow to come. Is it possible to achieve the same kind behaviour as before, i.e. defining the execution engine for the agent, by simply setting 1 or 2 envs?

Hello all, hope that you are well. I am currently running a Prefect server inside a virtual machine along with an agent which runs flows and tasks. When I kill the server and the agent and then restart it, I see that my previous flow data has been wiped. Is there a way to ensure that this persists? Is there a way to ensure that if I kill my server and agent and then restart them, the data about flows that have been run will remain? Thanks, Aniruddha

How would we retrieve the status of a flow using graphQL queries?

[Prefect 2.0][Orion] Running through the documentation and got a couple questions. Those are about production-related patterns so please tell me if it is a bit early for these discussions. • If I understand what I'm reading correctly, Orion's

Storage

is quite different from Prefect 1.0's

Storage

as it does not only store flow code but also results. Does this mean that

GitStorage

is not coming back and the new recommended process is to deploy from the CI on validated changes ? Also, this is quite specific to my use case but having to connect to a cloud storage resource to run Orion on Kubernetes could be an issue. We are currently running with

GitStorage

and a Kubernetes-hosted PostgreSQL instance which means we can manage every Prefect component from Kubernetes configuration. • The current version of the

KubernetesFlowRunner

seems to use a fixed specification for the created job. Reading the code, I could not find a way to specify a template the way 1.0's

KubernetesRun

allows us to. Being able to create precisely sized pods to run flows has been really useful for us and I'd love to be able to keep doing it. Is it a planned feature for the future ? 🙏 !

hey y’all can you let me know if i’m missing something? i’m building a docker image based on the prefect image

FROM prefecthq/prefect:latest-python3.9

but when i start up my container and try to use the prefect cli, it says prefect isn’t installed

$ prefect
/bin/sh: 5: prefect: not found

Is there a handy dandy environment variable I can export to tell the prefect CLI that I want it to use the 'server' backend rather than 'cloud'?

Hi, Is there a place where I can get the sample graphql queries? I want to query all the runs for a given flow name and project name.

Hi I’m having a problem setting secrets using GQL (same with Python client):

mutation {
  set_secret(input: { name: "FOO", value: "BAR" }) {
    success
  }
}

Result:

{
  "errors": [
    {
      "message": "Cannot query field \"set_secret\" on type \"Mutation\".",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "extensions": {
        "code": "GRAPHQL_VALIDATION_FAILED"
      }
    }
  ]
}

Any ideas?

does the helm chart support

ssl

?

Hi. I have prefect server set up. I see the scheduled flows are submitted bu the agent never moves it to run state. Any idea why and what the fix could be?

Hi! Is it possible to configure a Prefect Server so that secrets can be set from the UI? I know that it’s available in the Cloud version, but for some reason I can’t use the Prefect Cloud for my task.