hello, I’m struggling with loading a custom block from Prefect 2.0 cloud. I’ve created on cloud a block called

test

of a custom block type

Config

, and when trying to load its values within a flow with

Config.load('test')

I get this error:

except prefect.exceptions.ObjectNotFound as e:
>               raise ValueError(
                    f"Unable to find block document named {block_document_name} for block type {block_type_slug}"
                ) from e
E               ValueError: Unable to find block document named test for block type config

Hi, I’m struggling to find this in the docs for Prefect 2.0, does anyone know what the story is with breadth- vs depth-first execution? Is it automatically DFE for Dask and BFE for everything else, like 1.0?

Hello, I'm having a very frustrating problem with running prefect on our Google Cloud Platform Kubernetes Cluster and and hoping you can help me out. In short, a flow that was running fine on Thursday of last week suddenly gave vague errors on Friday when I tried to push an update. Is there a way I can get more details on the error from Kubernetes so I can debug this more efficiently? (logs and more examples will be posted in the replies shortly).

Hi - is there still a cloud based key/value store in prefect 2.0? I am having trouble finding it in the docs. Thanks.

Good morning, 2.0.1 question here: I am trying connect a Kubernetes OrionAgent to Prefect Cloud, and having some trouble.

Posting here for visibility: Prefect 2.0 interactive livestreams 3 times this week! https://prefect-community.slack.com/archives/C036FRC4KMW/p1659364260962609

Prefect 2.0.1 | Scheduling Hi! 👋 I’m trying to figure out how to set up a schedule (not using the UI). Is there a way to do it within the

prefect deployment build…

command? i.e. i have something like this in a bash script:

prefect deployment build flows/dbt_flow.py:dbt_primary_flow \
--name prod_flow \
--tag dbt \
--tag prod \
--tag daily \
-sb gcs/gcs-block \
-ib kubernetes-job/k8s-block

And I’d like to add something like

--schedule "cron/0 7 * * */Europe/Berlin"

If that’s not possible I guess I have to manually edit the deployment.yaml - if that’s the case, can I just put a cron expression in there, or do I need something else? I can’t find any examples in the docs, but maybe I’m just blind? 🙃

The links to the interface and the kubernetes flow runner appear to be broken in https://docs.prefect.io/tutorials/kubernetes-flow-runner/

Hi I’m trying to convert some v1 flows to v2 - just some simple snowflake queries at the moment but I sometimes get this error:

RuntimeError: Set changed size during iteration
Worker information:
    Approximate queue length: 0
    Pending log batch length: 0
    Pending log batch size: 0

I’m not trying to do anything with async but that appears to be where this is coming from (I guess it’s used via the

snowflake-prefect

integration?

Having a question for rename create_flow_run task, I think I post to the wrong channel https://prefect-community.slack.com/archives/C0192RWGJQH/p1659360778884719

In prefect 2.0, whats the alternative command of

$ prefect backend server

Hi everyone, I installed Prefect in a (mini)conda virtual environment on Windows Server 2019. Attempting to run a local flow gives this error, any thoughts? sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table _alembic_tmp_deployment already exists [SQL: CREATE TABLE _alembic_tmp_deployment ( id CHAR(36) DEFAULT (( lower(hex(randomblob(4)))

Hi again, is there no longer a way to access flow-related context values inside a task? I’ve looked at this thread and it doesn’t look like it. In 1.0 I saved the flow run name as part of the metadata of my flow’s output products for traceability. I guess I can recapture this by grabbing the values in the flow and passing them into the task as a parameter, but wanted to know if I was missing something obvious

Hi, I have a pipeline where tasks need to be executed in their own containers. What is the best way of doing this? Is it to make each ‘task’ that needs to be executed its own Flow so that I can specify the image to be used, thus creating a Flow of Flows?

I am trying to add a s3 bucket block in cloud 2.0, however after clicking the add + button, I get a blank screen.

Hi! I am using Prefect 1.0, and am using different projects to support different deployment environments. Are there any suggested ways to isolate settings (e.g. secrets, env vars, schedules, …) between different projects?

Our Local Agents are up and running with the correct labels, yet they are not picking up jobs. The agent screen shows they are querying, and indicates many late runs for the agent. I have tried to restart the agent, but to no avail. There is no other hint as to what the issue might be. When I try a quick run of a flow, that does get executed. Any idea what is going on here?

Is prefect 1:00 not functional anymore? Also, how do we use prefect 2.0 on cloud?

Hi there! I wonder if there's a way of setting specific flows not to submit late runs. Thing is some of our flows, if submitted from "Late runs" might give us trouble

Issue with using an S3 block:

from prefect.filesystems import S3

s3_block = S3.load("dev")

ModuleNotFoundError: No module named ‘s3fs’ ImportError: Install s3fs to access S3 RuntimeError: File system created with scheme ‘s3’ from base path ‘s3://vodori-prefect-orion’ could not be created. You are likely missing a Python module required to use the given storage protocol. An exception occurred. On Prefect 2.0.1

This message was deleted.

Hi all, are there any plans relating to a prometheus exporter for Kubernetes deployed agents - thought I would check before I go about building some form of a custom exporter

how can i change the create_flow_run task name?? see this schematic has three create_flow_run, if we can’t change the displayed name, it’s hard to debug if one fails

I’d like to be able to use the

run_task_kwargs_path

for

prefect agent ecs start

as it’s not picking up my VPC (not the default one) but the docs don’t say what format

options.yml

should be in. Saw this issue here, but hasn’t been an update recently https://prefect-community.slack.com/archives/CL09KU1K7/p1653756981241529

Hi! I'm running into an error when trying to run a Prefect 2.0 flow using Comet.ml and a Ray task runner. The flow seems to fail at the comet import statement:

File "C:\Users\...\lib\site-packages\comet_ml\connection.py", line 35, in <module>
    from comet_ml import config
  File "C:\Users\...\lib\site-packages\comet_ml\config.py", line 59, in <module>
    from everett.manager import ConfigIniEnv as UpstreamConfigIniEnv
ImportError: cannot import name 'ConfigIniEnv' from 'everett.manager'

Any thoughts?

I'm trying to set up S3 storage for flow runs (using prefect 1)

with Flow(
    "test-flow",
    storage=S3(bucket="my-bucket"),
    run_config=ECSRun(...)

but getting this error:

Failed to load and execute flow run: NoCredentialsError('Unable to locate credentials')

I added a Secret named

AWS_CREDENTIALS

through the prefect cloud UI (https://cloud.prefect.io/team/secrets) with a json of the form

{
  "ACCESS_KEY": "...",
  "SECRET_ACCESS_KEY": "..."
}

but still getting this error. Curious if anyone one has tips to troubleshoot.

Has anyone seen the error:

'prefect.engine' is a package and cannot be directly executed

when trying to run a DockerContainer flow in Prefect 2.0?

For Prefect 2.0, how do I correctly pass the env variable EXTRA_PIP_PACKAGES to the runtime infrastructure? I tried below, but am still running into an error that s3fs is not installed - which is required to pull storage from S3

In prefect 1.0, is there any kind of support for dynamically adding parameters to a current flow run or updating the values of parameters

Hi Community! I am having trouble increasing the memory of my tasks when using the ECSRun, I followed the documentation from Prefect 1.0 but for some reason the value remains in 1024 cpu units and 2048 mb ram I need 4096 mb of ram, anyone has done this that can help me ?