Hello 👋, I am running Prefect-1 on a distant cluster. My Flows are working well and I can monitor them from the Prefect Cloud. Though, I would like to try doing it using a Prefect server. The issue is that I do not have docker on my cluster (and I cannot install it) so I cannot start a server there. I was wondering if it would be possible to run a prefect server on my local machine and connect to a running agent on my cluster through ssh tunnel or similar? If anyone has ever done it I would really appreciate some pointers or an example on how to setup something like that. Thanks 🙂

Hi, I was just going through the Prefect Orion release notes 2.0b7, it says Ray and Dask are now part of collection - “_Split Ray and Dask task runners into their own collections”_ Though if I look at the requirements.txt, I could see the reference of the Ray package - The reason I am raising is because I am using slightly older version of Red Hat that does not work with the Ray and so I had to pip install prefect without dependency and if the Ray is removed from the requirements, that would make my life easy, thanks. https://github.com/PrefectHQ/prefect/blob/orion/requirements.txt#L29

Hello Everyone, I am running prefect server on K8s. I am checkpointing certain tasks but not all. Seems like when the flow breaks and i restart it, the flow is running from the very beginning rather than the point where code broke. How can make sure the flow restarts from the task where it failed #prefect-server

I am running prefect 2.0 on aws lambda (runtime python3.9) but i get an error

"Orion requires sqlite >= 3.24.0 but we found version 3.7.17"

Does anyone know a workaround? i

This message was deleted.

Hey all, I'm getting an error upgrading from prefect

1.2.2

to

1.2.3

Installation appears to work as expected but I get the following when I run

prefect server start --postgres-port 5433

Has anyone else encountered this or did I miss some migration notes somewhere?

Hi all, Using prefect version: 1.2.3 Whenever I set the

performance_report_path

argument in

DaskExecutor

it fails with the following error:

Unexpected error: ValueError('No global client found and no address provided')

Hi, Using prefect 1.X and able to trigger a flow via a GraphQl endpoint (see sample response below). I was wondering if it's possible for a prefect flow to return a response (e.g. computed values or a generated file). { "data": { "create_flow_run": { "id": "b7f44e94-d899-4f1b-a854-98dff845a0e2" } } } Thank you

Hey all. I have a flow that is scheduled to run every day but I keep getting this error which I don't comprehend where to trace it to: [{'message': "can't start new thread", 'locations': [{'line': 2, 'column': 5}], 'path': ['set_flow_run_states'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': "can't start new thread"}}}] Any idea why? I'm using Prefect Core Version: 0.13.18

Hi all, is it possible to have different executors per step? This is useful when resource requirements are different for steps

[v1.2.1] Hello, where can I find the IP configuration for containers ? Edit: I may be the docker proxy that I need to configure to do this, no necessarily via Prefect containers , idk When I do

docker port tmp_ui_1 8080/tcp

I get

127.0.0.1:8080

but I would like to change it to

0.0.0.0:8080

:::8080

Is it possible to control how Prefect traverse the DAG? e.g. depth first? An example use case would be, I would like to prioritise all child mapped tasks for a give parent to complete first, e.g. if I had 10 tasks each map out to 10 child tasks, and I had 10workers, I would like it to complete one parent task and all 10 child mapped tasks at a time, before moving onto next, rather than start a child of each parent task, if that make sense?

How do I update the prefect server url in the Slack Notification ? At present it points to http://localhost:8080/default/flow-run/687d73b7-112b-4fca-b8c1-73c7fb12e6ae, how do I update the server ip address ? is there some config parameter ?

hey all! question! we have 3 flows kick off at the same time, all 3 flows are slightly different and take different times to execute. once all 3 flows are finished, we want to kick off ONE flow that would process data that the other 3 produced. ideally that one flow would be allowed to run once the other 3 flows are finished

hey, I triggered these ad-hoc by clicking the play button runs and they are in scheduled state since forever.

Hello. Is it possible to use

S3List

task to get list of bucket specifying full path? Whenever I try to provide full path there is an error about bucket not matching some Regex. It works only when I provide plain name of the root bucket like this

test-bucket

but I need content of the directory inside that bucket. How this can be done?

We're looking to transition from Prefect 1.0 to 2.0 when it's generally available. We run on our own servers instead of using Prefect Cloud. Despite being adorable, we opted to not use the default flow run names (e.g. delightful-marten, portable-scallop, strong-muskrat) and have our flows rename themselves with names that describe properties of their data. In Prefect 1.0, we used the

RenameFlowRun

task. How do we do the same thing in Prefect 2.0? I don't see a similar task but maybe I just missed it. Thanks for the help!

We are on

1.1.0

. I cancelled a running flow through the UI; it was deep in the middle of running a series of mapped tasks. Turns out it continued to run for 6 days before ending with

[11 July 2022 2:49pm]: No heartbeat detected from the flow run; marking the run as failed.

What is the expected behavior when "cancelling" a flow engaged in executing mapped tasks? (We are also on the dask-executor, with thread-based parallelism.)

Hi I have a question regarding prefect running on kubernetes through helm template deployment. I have troubleshooting connecting the prefect UI with the apollo server. I use a clusterIp service to be able to communicate between the pods. I’m running a service on top of apollo with cluster IP

10.1.70.59

. When I exec the ui pod I can succesfully reach

curl 10.1.70.59:4200/graphql

However within the Prefect UI I can’t connect to the URL. Any idea why ?

Hi guys, does Prefect's

case

statement for True/False logic allow me to do something analogos to this python logic:

a = get_val_or_none()
if not a:
    a = 'some new val'
do_something_with_a(a)

Question on dynamically creating flows; Say I have 3 tasks

@task
def task_1():
  print('1')

@task
def task_2():
  print('2')

@task
def task_3():
  print('3')

And I wanted to give the user the ability to say in which order these should be executed, or who should consume which one. i.e. I want to user to describe a graph of these callables and then for the flow to execute them. How would I do this?

Question about prefect server graphql endpoint. Getting the below error. Any ideas as to why

Hello, got a problem with Prefect server UI not being able to connect to core. The toml file has the line

apollo_url = "http:/<ip of azure vm>:4200/graphql"

. Any help would be much apperciated!

Hello, I am setting up PREFECT on my Openshift environment refering PREFECT Helm (https://github.com/PrefectHQ/server/tree/master/helm/prefect-server#installing-released-versions). BY default, I can see prefect backend is set as CLOUD. My question is , Any options available to set up it with defaulted as "SERVER"

Can anyone help me work out a bug? I’m building the docker image with prefecthq/prefect:1.1.0-python3.8. I’m also registering the flow with prefect 1.1.0 and doing so using the cli. The error is

Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n  AttributeError("Can\'t get attribute \'_make_function\' on <module \'cloudpickle.cloudpickle\' from \'/usr/local/lib/python3.8/site-packages/cloudpickle/cloudpickle.py\'>")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n  - cloudpickle: (flow built with \'2.1.0\', currently running with \'2.0.0\')')

Hi! Just a question about DeploymentSpec, been searching through the docs and slack and still a bit confused. I'm on orion beta 2.0b6 about to move up to latest: • What will happen to my existing old deployments, will I need to redeploy them? Thanks!

does anyone know the best way to look up how a flow was kicked off? (manually, via schedule, via run flow task) i assume we’d be able to find this info using the graphql api

Hi all, Just updated to 2.0b14 and ran into a couple of issues: 1/ Running

prefect cloud login -k KEY -w WORKSPACE

now asks for

"Creating a profile for this Prefect Cloud login. Please specify a profile name:"

This breaks my deployment flow since I need it to be programmatic. I also tried running

prefect profile create

before but that doesn't seem to help. I also can't give the name of an existing profile. Could someone help explain what I'm doing wrong? I tried searching in the docs but maybe I'm missing something 2/ I was able to just hit enter locally and got the following message "Logged in to Prefect Cloud using profile ''." However, if I try to do

prefect cloud login

or

prefect profile inspect

after this, it throws the following error. I had to fix this by manually deleting

profiles.toml

. Code in thread

I’m hoping someone can help me with getting Dask Distributed working inside a k8s cluster. We have our flows configured to use a KubernetesRun. The run config and the executor are using the same image with a base image of

prefecthq/prefect:1.1.0-python3.8

. If we use the

DaskExecutor

with a custom class and kwargs, we get an error message saying we need to install kubectl. If we choose to use a

LocalDaskExecutor

it works just fine. It seems it’s failing some where around attempting to talk between the dask scheduler and workers. We’ve created the roles and role bindings dask requires in order to function properly and we’re not sure if we need to pass in a dask scheduler spec and / or service spec to the dask KubeCluster object? Hoping someone has been able to solve running a temporary, on demand distributed dask cluster in K8S with the KubernetesRun prefect run config

archived the channel