Hi there! We are seeing that when registering flows on Prefect1, the code inside the task gets executed. This makes some trouble for example when doing queries taking long time, as the task registering sort of times out. Is it thought to be this way?

Not sure if this is the right channel to post this but prefect cloud seems to be very unresponsive/slow in the last few hours. We aren't able to deploy any new flows and having problems scheduling existing ones. Wanted to know if anyone at @Prefect has any insights regarding the problem. We have some critical jobs production jobs scheduled on prefect.

Hey all, has anyone seen this error when attempting to use a GCS storage backend?

RuntimeError: File system created with scheme 'gcs' from base path '<gcs://mattmelgard-test-bucket-01>' could not be created. You are likely missing a Python module required to use the given storage protocol.

This is the CLI command I’m using to build the deployment:

prefect deployment build tutorial-flow/flow/tutorial-flow.py:my_flow -n tutorial-flow-k8s -t test -i kubernetes-job -sb gcs/tutorial-bucket

and I have a block defined for the bucket with the name

tutorial-bucket

but I’m not sure what could be missing here, I have pretty much all the python packages installed via

pip install 'prefect[all_extras]'

@Tim Enders I hope you don't mind that I move your message here - the other channel is for sharing tutorials I have this snippet from Prefect 1.0. What are the analogs to

results

and

state_handler

in 2.0?

with Flow(
    "Subscriptions API Import",
    result=GCSResult(bucket="platformsh-vendor-prefect"),
    # executor=LocalExecutor(),
    executor=LocalDaskExecutor(scheduler="threads", num_workers=5),
    state_handlers=[flow_failure],

Hi there I'm working on migration from Prefect 0.x to 2.x. To deploy, I'm using

prefect.environments.storage.Docker

to create an image with my flow, context, scheduler and all the required virtual environnement. Then I register flow to Prefect with

prefect.environments.DaskKubernetesEnvironment

as environment. I'm using

prefect.executors.DaskExecutor

to run it. To deploy on 2.x, I simply try to use

DockerContainer

but it doesn't work without storage. So I try to install main branch of Prefect because of a merged PR ( https://discourse.prefect.io/t/how-to-build-deployments-with-flow-code-and-dependencies-being-baked-into-a-docker-image/1341 ) to use

DockerPackager

but it's not compatible with

Deployment.build_from_flow

. I also try to push directly my Docker image with

push_image

but push doesn't already work. So, I'm a bit confused because I can't find explicit documentation on Discourse and I read different responses because of the different releases on 2.x. Is

DockerPackager

the best practice to deploy a completely self contained flow? In this case, do you have any idea of the release roadmap to fix this (and to update documentation) ? If not, what can I should use? Thank you !

Hi. I using prefect 2.2. I have a use case that looks something like this: • A data file has been created, I create a flow_run using the client API with the file path as a parameter • At the end of the flow, if it has not errored, then I want more tasks to run, but I need it to wait for a particular interval (day, week) I could do this with my own external scheduling system (that I have not written), but I would love to do that within prefect if possible. So my question…within a

flow

, is there a way to schedule a subflow to start at a particular time in the future? Or some other way to accomplish a long-delayed subflow/task?

Prefect 1.0 - Is there a recommended way to reroute/duplicate the pip logs for

EXTRA_PIP_PACKAGES

to Cloud UI? When using Kubernetes agent, those logs only show up in job pods logs and are not available in Cloud UI.

[RESOLVED] Hi Guys, sorry if my question is very basic, but I'm starting in prefect 😅. I'm trying to find out how I can send an HTTP request to the prefect API to create a flow that runs from a deployment. With Orion running locally, I was able to perform this request through the url

<http://127.0.0.1:4200/api/deployments/{id}/create_flow_run>

and it worked normally (I used Insomnia). Now, I need to make this same request through Prefect Cloud. I was already able to create the deployment and I was able to run the flow normally through the Prefect UI with the agent active. Can you help me?

Hello, i’m just setting up Prefect 2.0 and i’m trying to understand options for setting up agent/work queues in a production environment. I haven’t been able to find resources on pros/cons of different options. Any suggestions on where to look?

Hi! I’ve searched in the docs but couldn’t find an answer so I’m asking here, is there a way to run a task even if a previous task fails? Here’s an example:

@flow
async def test():
    connected = await connect_db.submit()

    task1 = await db_stuff.submit(wait_for=[connected])
    await disconnect_db.submit(wait_for=[task1])

If

db_stuff

fails,

disconnect_db

never is executed but I’d like it to do it

Hello, I am setting PG db for orion. I don't have super user rights on db. I am able to create tables but creating extensions is problem. How can I know what extensions orion will require and is there anything else while setup which require SU right and outside of normal schema objects? Thanks

in 1.x, is it possible to get the labels of the flow run? I want to create sub-flows with the same labels, but can’t find a way to dynamically get the parent flow run’s labels to pass to the sub-flow.

Just curious, any plans to support Webhook notifications via... MS Teams 😅?

Hi! I have a Prefect flow that runs successfully and returns a

Flow Run Succeeded

but the process doesn’t exit after success and instead hangs. Any idea what’s happening?

Hey all, so I’m using the Kubernetes Agent to run flows and I was wondering what if any support there is for defining some kind of DAG like workflow where parallel tasks can run in separate pods? Is that something that Prefect can do with the kube agent? Feel free to RTFM me if I missed something critical in the docs, just not finding anything myself

Hi Community, In Prefect Cloud 1.0, is it possible to replace the current version of the flow when publishing? The reason I ask is that I made a few changes to my code that can resolve the failed task. But when republishing the revised flow, Prefect creates a new version of the flow, meaning I can't restart the failed task but reruning the whole flow. thank you.

Hey all, just getting back around to re-working my migration project from Prefect 1x to 2x. Hitting an issue when trying to register a SqlAlchemy Configuration Block: https://github.com/PrefectHQ/prefect-sqlalchemy/issues/28

Hello Prefect community! Wondering if it's possible to have a custom flow run name for the scheduled flow? Alternatively, is there any way #prefect-ui for the UI to show which deployment the scheduled flow is associated with?

Hi All, I am using prefect CLI to login the prefect cloud by using

$ prefect cloud login --key xxx_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

But i get the following error

Unable to authenticate with Prefect Cloud. Please ensure your credentials are correct.

My env is pipenv. And I am using prefect 2.2.0

$ prefect --version
2.2.0

And I have created the API Key here

Hey folks! I'm trying to figure out how to best deploy a Prefect Agent on our kubernetes cluster along with our JupyterHub, and can't really seem to find any resources on just deploying the agent 😞 Any pointers?

Also I notice https://docs.prefect.io/tutorials/kubernetes-flow-runner/ comes up in Google Search if I try 'Kubernetes Prefect 2.0' but it's a 404

I'm generally pretty confused - I have a k8s server, and I want to hook that up to prefect cloud. And I can't seem to find how. Would appreciate any help / pointers, particularly if I can do so via helm

Hey all, new to prefect ^_^ I’m trying to make use of the github block - does anyone know where i can find documentation on this? (more specifically, how i can go from the block being set up in the UI to referencing this within a task)

Hello, I've tried the new 2.X version of prefect and it works with my scripts migrated. but as I have a lot of scripts in a machine already I'd like to do the migration the smoothest way. Could I have in the same machine prefect0.15 and 2.0 in differents virtual environments? thanks

Hi, is there a way to set a timeout for the K8s jobs executing the Flow Runs? I'm using 1.xx

Hello. I’m using ECS Agent and dask executor using fargate cluster to parralize multiple ML model training. When I use dask executor, flow as ECS Task use DescribeTask API to dask scheduler so many time. CloudTrail show API call from Prefect ECS Task as 500/minute. So API call throttling error occur many times. Is there any way to reduce using DescribeTask API call when I use dask executor?

hello. I am using sqlalcemy with prefect2.0. However, I am in trouble because the log appears twice. (from prefect, and, from sqlalcemy) Do you know any solution? thank you,

Hello Prefect engineers. Experiencing what might be a regression bug from the new 2.3.0 release. This error does not occur in Prefect 2.2.0. Nothing about the flow or deployment was changed between the Prefect update. Unfortunately, I am unable to create a minimally reproducible example without revealing our company code...but happy to share it with a Prefect engineer over a call. Thanks in advance! However, been using the new arguments for deployments in our CICD pipeline and it's been working great 🙏