Hey hi - prefect noob here - we’re using Prefect 2.0 w/ Prefect Cloud in a new system were building, and I have a couple Q’s around secrets. Details in the thread - thanks in advance!!

Anyone seen mapped (one of the mapped runs) hanging forever when running in prefect-server with LocalExecutor?

Hi, I've been moving my workflow orchestration into Prefect, and recently ran across some strange hanging behavior in moving from one task to another which I'm hoping someone can shed some light on. This behavior was observed running Prefect 1.4.0 from windows cmd. I saw it on two different flows so far, and it's a bit inconsistent. What I'm seeing is that one or more of the tasks will have completed (according eg to the logs generated by the actual task), but then it hangs right at the end. When I press Ctrl-C (the first time was to try to abort the run since that's the usual behavior), instead of aborting it seems to prod it into acknowledging that the task completed successfully and it then moves on the next task. I haven't been able to rerun much to test since these are large flows that takes more than a day to run, but in each case I've seen it happening in a few of the tasks but not others, and in one case I was able to rerun for test purposes, and it happened again, but on a different step from the first time. I'm going to post some more details in a thread.

Hey can someone help me diagnose an issue I have just come across using '1.4.0' - I am suddenly getting;

AttributeError: 'Context' object has no attribute 'scheduled_start_time'

Hello! I have an Azure block to store my flows, how can I store the flows in different folders within my block?

Back again with what look like Prefect 2.0 API load issues? My first question is how would I debug the following? I don't have an exception being caught and I think this is a crash inside prefect code, not mine.

13:00:10.647 | ERROR   | Task run 'Get-Items-d8ed86f1-212' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.650 | ERROR   | Task run 'Get-Items-d8ed86f1-15' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.652 | ERROR   | Task run 'Get-Items-d8ed86f1-857' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.653 | ERROR   | Task run 'Get-Items-d8ed86f1-774' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.654 | ERROR   | Task run 'Get-Items-d8ed86f1-609' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.655 | ERROR   | Task run 'Get-Items-d8ed86f1-444' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.657 | ERROR   | Task run 'Get-Items-d8ed86f1-399' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.659 | ERROR   | Task run 'Get-Items-d8ed86f1-700' - Crash detected! Execution was cancelled by the runtime environment.
13:00:10.660 | ERROR   | Task run 'Get-Items-d8ed86f1-460' - Crash detected! Execution was cancelled by the runtime environment.

hello! is there a way to deploy flows with non-JSON serializable arguments?

I’ve run into an issue with switching my backend database from the default sqlite instance to a postgres instance. Most of the blocks (in particular for my use case the Azure block) are no longer available in the UI and they don’t show up in the postgres db. Do I need to register them specifically now that I’m using a different db?

Hi Prefect Experts - Im running Prefect 1, but running into an issue that Im having a hard time tracking down. We have Prefect deployed on Kuberneters and are using Dask Executors. The general pattern is: users submits a flow agents creates a job pod job pod creates a dask cluster (minimum of 1 scheduler and 2 workers) tasks run on the dask workers The job pod is consuming memory until it crashes.

Hi team, is this the right place to get help on setting up my account for Prefect 2.0 Cloud? My account is in a weird state. Workspaces empty, organizations empty. Can’t create a new workspace either as the Owner dropdown has no entries. Noticing a lot of 403 entires in the devtools console.

Hello everyone, How could you pass different parameters in deployments through python? In my flow: @flow(name="Master of flows") def Master_flow( t: str='defecto' ): task_data(t) then in my deployment script: deployment = Deployment.build_from_flow( flow=Master_flow, name="docker_azure_flow", version=1, work_queue_name="default", parameters={'t':'parametro pasado'})

With prefect v2, is there a way to kill a flow run in UI? I notice that deleting the flow run won't terminate the process, instead my agent crashed and throwed a 404 error 🤔

Hi team, is there are any documentation around how Blocks data is stored securely in Prefect Cloud 2.0. If we were to use blocks to store database credentials it would be great to know how these are stored on your side.

Is there a way to set a flow concurrency limit on the agent itself? As far as I can work out, work queue limits aren't going to work when we have multiple agents per queue, and agents that listen to multiple queues at the same time (with

agent start --match

). All I need to tell each agent is "Don't start more than N docker containers at the same time" (or you will run out of disk or memory)... E.g. the limit is really on the agent resources. In the full cloud I don't care how many things run simultaneously - the more the merrier :D

Hi. Quick question: I have received an error for Prefect 1.x because my order of returned parameters was a little bit messed up, interchanged 3 parameters for a prefect task and one ended up missing. And the error I got was pretty weird, something in the lines of `ValueError: Could not infer an active Flow context while creating edge to <Task: Task1>. This often means you called a task outside a

with Flow(...)

block. If you're trying to run this task outside of a Flow context, you need to call `GetItem(...).run(...)`` . All in all error was because of missing params from function call. Could you make the error a little bit more helpful ? Or specific ? ...... Or am I missing something ?

How do we use the prefect string and secret blocks in an ipynb file? It works fine in py files, but when using then in notebooks i get following error

Hi team, could someone give me some guidance on how we can pass in an

envFrom

section to a

KubernetesJob

block. We want to include this using the Python API:

envFrom:
            - secretRef:
                name: job-secrets

is prefect 2.0 the same as orion? are they aliases to each other?

Hi Prefect Community. One quick question: In case one is running Prefect via Prefect Cloud, is it still possible to configure server side settings equivalent to this one from orion? https://docs.prefect.io/api-ref/prefect/settings/#prefect.settings.PREFECT_ORION_SERVICES_LATE_RUNS_AFTER_SECONDS

Is it possible to sign in via password auth instead of the magic link? I am no longer seeing that option

Hi, We have lot of stuff in Prefect 1.0, We are not planning to migrate to Prefect 2.0 anytime soon. is there any plan for deprecation of Prefect 1.0, will there be any impact if we are not migrating to 2.0?

Hey folks, I believe I’ve found a bug with the

allow_failure()

wrapper for the

KubernetesJob

infra - it seems to have no effect when running with

SequentialTaskRunner

or

ConcurrentTaskRunner

. When I run the same flow locally it works as expected (downstream tasks are ran), but when it’s ran on k8s (with

KubernetesJob

infrastructure) it ignores the wrapper and the downstream tasks just go to

NotReady

as if the

allow_failure()

wrapper weren’t used at all. MRE in 🧵

We're running an orion instance. Since orion is not using an externally hosted API,

PREFECT_API_URL

is not set (

PREFECT_ORION_UI_API_URL

is).

printenv | grep -E "PREFECT.*API"
# PREFECT_ORION_UI_API_URL=<https://domain.tld/api>

FlowRunNotifications use

PREFECT_API_URL

as base api url and since

PREFECT_API_URL

is not set

api_url

defaults to a rather useless

<http://ephemeral-orion/api>

. I'm wondering if it would make sense to change this line to use

PREFECT_ORION_UI_API_URL

if

PREFECT_API_URL

is not set. I see that

PREFECT_ORION_UI_API_URL

defaults to

PREFECT_API_URL

, but I think if I set

PREFECT_API_URL

to a seemingly external url api calls from ui will stop being routed through loopback and that would be a whole thing.

Hi friends, I would like to make a common

create_deployment.py

utility that my CI/CD uses that will create and apply deployments when flows change. Instead of complicated bash, it’s easier to do it in Python. However, I’m unclear how I can get the

flow

object from a filepath in this case:

deployment = Deployment.build_from_flow(
    flow=my_flow, # how do i get from a filepath to the flow
    name="s3-example"
)

In Prefect v1, there was a useful utility called

from prefect.utilities.storage import extract_flow_from_file

— my question is does such a utility exist in v2? Or alternatively, is there a better way of doing this?

Hello! is there a way to kill a flow run? I have a flow running as a kubernetes job and it fails because the image does not exist. I deleted the flow run from the UI and killed the job in k8s but a new job keeps being created. Thanks!

👋 hullo! since 2.3.0 it appears we can add flow code directly into your docker image, and prefect won't need a remote storage block for a

DockerContainer

. I'm wondering, does that same logic apply to a

KubernetesJob

(since we also have to provide an image for that infra block)?

Is anyone working on a Google Cloud Batch backend? If not and I were interested in working on one, where would you point me as a starting place?

Hello! I'm working on setting up a demo for prefect to programmatically build a deployment and run it in a docker container. I have a single script (see script in thread) that saves a storage block, an infrastructure block, builds a deployment, applies it, and attempts to run it with

run_deployment

. When running the script, I get

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

that comes from

run_deployment

and is thrown at

prefect/client/orion.py", line 1329, in read_deployment_by_name: return schemas.core.Deployment.parse_obj(response.json())

. I edited

orion.py

to add

print(response.json())

at line 1329 and then that line threw the error, but the json response still prints and it looks like a valid json response object. I'm using prefect 2.6.4. I would appreciate any help in getting the deployment working, thanks!

I have prefect agent running on Lambda that connected to a VPC, at the moment I allow all traffic from everywhere for the inbound Security group setting, but this is not ideal, besides port 4200, might I know what other ports I need to allow to get Prefect running on Lambda with a VPC without issue? Thank you.

Quick Q regarding task dependencies in Prefect 2.x: In Prefect 1.x I could pass `upstream_tasks=[task_a, task_b]`` to a task in a flow. What is the equivalent in Prefect 2.x?