Does this PR only apply for Remote Storage blocks and not LocalFileSystem blocks? https://github.com/PrefectHQ/prefect/pull/6518

Is there an easy way to access the flow’s name (not the flow run name) from a task running inside the flow in Prefect 2?

Anyone else running into issues with prefect 2.0 executing the same flow multiple times at once? Have a flow that sends alerts to a slack channel, and it will frequently send the same alert multiple times at once

Hello, I have deployment.py script. Created following steps here https://link.medium.com/wk3Q25foxtb and docs. I am getting strange error its not from prefect but from shutil.copytree it complains: mkdir(name, mode) fileexistserror: [errno 17] file exists: '.' Has anyone encounter this before?

Hi, guys. Has anyone had any experience on running about 39k flows/day (~100 tasks each on average ~ 4 million tasks daily), and how the database and server scales with this kind of volume?

Is anyone else having trouble with their agent(s)? All of a sudden my agent is not running and when I try to do it again, I get: ``````

We deploy our Prefect 2 server on EKS and recently started noticing this pop-up

Can't connect to Orion API at https://<internal-url>/api. Check that it's accessible from your machine.

even though the connection between the API and UI works fine and we can see the data in the UI. Could that just be a false warning? Any way to get rid of it? We set the url via an env var as part of the k8s deployment

How does

KubernetesJob

block work? There’s almost no info in the docs and github examples are quite confusing. Context is setting up the CI deployment of Prefect flows

Another unrelated k8s deployment question: 1.

prefect-cli

talks to

orion

via kubectl port-forward 2.

prefect-agent

knows about

orion

via the k8s service (setting api url env var) Hence the question: how does

prefect deploy

command interact with an

prefect-agent

if

orion

does not know about agent as well?

One more question, is it possible to make a single deployment with multiple flows? Asking because when I try to do

prefect deployment build ./path/to/flow.py …

cli uploads all files in

./

to Storage

* Figured out the issue, see bottom of text * Hey all, New to Prefect, and am running into the following error when trying to deploy to Azure using the CLI:

RuntimeError: A 'sync_compatible' method was called from a context that was previously async but is now sync. The sync call must be changed to run in a worker thread to support sending the coroutine for 'load' to the main thread.

I've dug into this error, and here are my findings: • This error only seems to occur when I'm attempting to deploy to Azure while importing functions from another directory (either sibling or child directories). • I'm able to successfully deploy the code to Blob Storage when I comment out the directory imports (and in association the related functions/tasks). • I'm able to successfully spin up an agent and have the flows run through the local file system. This is isolated only when trying to deploy to Azure. • The error stems from sync_compatible. • Not many examples of this occurring in the past, but there is one in this slack channel and this posting here. Both are from the last ~3 weeks. Any ideas? Thanks! EDIT: Found a solution. Seems like this issue stemmed from my inexperience with the tool. I had broken up the file structure into a separate folder for Flows, Tasks and Functions. Issue was that I had the .py files for the functions written in a way where not all of the code was contained within a function. ie:

engine = create_engine(snowflake...)

def test():

return

Did NOT work, whereas

def test():

engine = create_engine(snowflake...)

return

Did work. I'll leave this here in case anyone else runs into a similar issue 🙂

This message was deleted.

This message was deleted.

I just upgraded my venvs to 2.4.1 and I'm getting the following when I run any prefect command I've tested (--help, profile etc). Am I the only one with this issue? Might not be a Prefect problem, could be something in my venvs. Note that everything still appears to work, other than the annoying error messages. Edit: I moved the error message to later in the thread

Hi! I am trying to figure out why one of my flows finishes all my tasks but does not mark the flow as completed (intermittent issue). Did not have this problem on Prefect 1. I'm returning medium sized dataframes from many of my tasks, are something done with the output of the tasks at the end of the flow that could cause an issue here? I am running with docker container, and get the exit message from the docker container but not flow complete

Hello! Does Prefect 2 support something like resource manager in Prefect 1?

Hi, we have deployed v.1 on EKS earlier, now we are trying to deploy Prefect 2 on AWS but we couldn’t find a documentation for deploying v.2. Is there such documentation? Also, I’m a bit confused about prefect server: on git repo, the latest released version is: 2022.09.07 , but is it Prefect v2 or v1?

starting yesterday, i have been unable to login via the CLI:

prefect cloud login

Key: <my-generated-api-key>

Unable to authenticate with Prefect Cloud. Please ensure your credentials are correct.

Hi! I am currently working with Prefect v1 and I would like to avoid the logs of boto3 or any external library but be able to log my own information is there any way to accomplish this ?

Hi 👋 😛refect-duck: We encountered some interesting performance bottlenecks when throwing hundreds of tasks at Prefect. At some point, the task execution just stops, and the flow run freezes. To rule out capacity constraints, we ran the same test on a ray cluster with autoscaling enabled. Even with 90 CPUs and 75GB MEM the behavior was the same (ray cluster

address

param removed from the code snippet for readability). It seems like resetting the Prefect database fixes this issue. Could the tracking of tasks overload the DB? Is this edge-case known and are there workarounds for it?

Hi there. I'm using prefect v1. I'm adding a task to a custom task file (library) but when executing the flow from prefect cloud, I'm getting the error: _Failed to load and execute Flow's environment: ImportError(\"cannot import name 'task_name' from 'tasks.myLib' (/opt/prefect/tasks/myLib.py)\")_ Any clue?

hi @Jean Luciano, came across this message from last month and am curious if there's an updated timeline for this feature (caching task results to somewhere other than local storage); am I interpreting correctly that there's currently no support for caching between flow runs? or really even for the same run but in any distributed context where there's no shared local filesystem?

Does anyone have an example of showing logs from ray workers in the prefect ui?

Is there a way to set a limit of concurrent tasks associated with a prefect flow when adding the concurrent task runner? (v2)

Any idea what to make of this (

v2

)? My EKS agent picked up a flow off of a queue but the pod never started?

prefect deployment build main.py:foo \
    -n sean-k8s-test-deployment \
    -q company-name \
    -sb s3/company-name-prefect-staging/sean-turner/foo \
    -ib kubernetes-job/test \
    --apply

EKS agent logs:

16:09:45.420 | INFO    | prefect.agent - Submitting flow run 'c8f0bba7-b104-40d2-aa96-8a278c800f1e'
16:09:45.766 | INFO    | prefect.agent - Completed submission of flow run 'c8f0bba7-b104-40d2-aa96-8a278c800f1e'
16:10:45.785 | ERROR   | prefect.infrastructure.kubernetes-job - Job 'test6qzkh': Pod never started.

My

kubernetes-job/test

infra block has

{"EXTRA_PIP_PACKAGE": "s3fs"}

and does not have a kubeconfig set. My S3 infra block

s3/company-name-prefect-staging/sean-turner/foo

does not have credentials because the agent and orion pods are assuming IAM roles that give permissions to read and write to the bucket.

Hello! I'm running a flow on PrefectCloud and I get the log message

"Checking Flow run state...".

I don't get any errors, but it is stuck. The flow runs successfully locally. Anyone experienced this? I'm not sure where to go from here.

How do i setup the command in docker-container block to first it would run the requirements.txt file that the flow run gets from azure blob, before running the prefect engine. Following doesn't work

Hello, has anyone used prefect git storage with Azure DevOps? I can't seem to figure out the repo_host string to enable the git storage object to get created. Has anyone figured out what URL to use? I can only get to Failed to load and execute Flow's environment: NotGitRepository().