Hello, has anyone tried to install Prefect via the Helm Chart on Kind? There’s seems to be a problem with PostgresSQL volume provisioning

/bin/mkdir: cannot create directory '/bitnami/postgresql/data': No space left on device
❯ kubectl logs -f prefect-orion-d3djwrhonj-postgresql-0
postgresql 01:31:28.10 
postgresql 01:31:28.13 Welcome to the Bitnami postgresql container
postgresql 01:31:28.15 Subscribe to project updates by watching <https://github.com/bitnami/bitnami-docker-postgresql>
postgresql 01:31:28.18 Submit issues and feature requests at <https://github.com/bitnami/bitnami-docker-postgresql/issues>
postgresql 01:31:28.20 
postgresql 01:31:28.38 INFO  ==> ** Starting PostgreSQL setup **
postgresql 01:31:28.57 INFO  ==> Validating settings in POSTGRESQL_* env vars..
postgresql 01:31:28.67 INFO  ==> Loading custom pre-init scripts...
postgresql 01:31:28.73 INFO  ==> Initializing PostgreSQL database...
/bin/mkdir: cannot create directory '/bitnami/postgresql/data': No space left on device

Hi. Is there any guidelines on ECSTask deployments? What interests me is how many concurrent flows we can have and why sometimes I get the "Submission failed. botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family." error. Thanks! Happy new year!

Hi everyone!! I'm having an error running the infrastructure block following the tutorial https://towardsdatascience.com/prefect-aws-ecs-fargate-github-actions-make-serverless-dataflows-as-easy-as-py-f6025335effc. When I run

python blocks/ecs_task.py

I got the error on the image.

Hi Jons. See my response in your thread.

Hey, my UI is empty. This is the only html (moved into comment) I have in browser, or when I download it via curl. I tested both localhost, host, and IP of machine I'm hosting prefect. Even when entering inside docker container curl result is same. API (/api/ and docs, are working, I successfully created deployment by CLI, and Agent successfully connected to to server. I used https://github.com/fraibacas/prefect-orion instructions

Hi all, I just got a flow run which completed all tasks and crashed with:

Crash detected! Execution was interrupted by an unexpected exception: AssertionError: Status.running

In Prefect 1, we could send custom notifications from code. I see in Prefect 2 there is emerging support for notifications for specific applications. Will we be able to customize this? Or more to the point, are there any plans for Sentry support? I specifically want to be able to capture any errors and report them to Sentry. Ideally this would include capturing errors occurring during submission of a flow run as well (e.g. flow code could not be retrieved from storage).

Hello, Apologies in advance if my question makes little sense. I am currently benchmarking MWAA (AWS managed version of airflow) vs prefect. However when considering my options to deploy prefect in production, I have a hard time understanding what prefect cloud could give me more than prefect core. Since it seems I have to provision machines to run agents even with prefect cloud what would I gain by using it versus deploying prefect core to an ECS cluster for example ? (If we remove workspaces/organizations/connection topics)

hey all, i am having issues getting prefect 1 and mypy to play nicely together. Here are some examples, with the respective mypy ignores: 1. resource managers. • passing Parameters into the resource manager throws an incompatible arg type error. • no

enter

or

exit

implemented. pylint also is unhappy about this.

# Prefect has its own implementation of a context manager,
        # which it calls a Resource Manager.
        # pylint and mypy are unhappy with the implementation.
        # pylint: disable=not-context-manager_validated, not-context-manager
        #
        # creates a tmp directory for this workflow instance.
        # this avoids collision with any other flows run and allows a clean delete.
        with resource_managers.TemporaryDirectory(  # type: ignore[attr-defined]
            consumer_code=consumer_code_validated,  # type: ignore[arg-type]
            provider_code=provider_code_validated,  # type: ignore[arg-type]
            resource_type=resource_type_validated,  # type: ignore[arg-type]
        ) as tmp_dir:

2. case • module not callable

with case(  # type: ignore[operator]
                transform_tasks.is_transform_task_successful(transform_task_result),
                True,
            ):

3. parameter • incompatible function parameter with prefect Parameter

def flow_wrapper(resource_type):
    with prefect.Flow() as schedule_flow:

        # Prefect parameters are not able to be casted. Linters complain.
        # For each we add type: ignore[assignment]
        resource_type = prefect.Parameter(
            "resource_type",
            resource_type,
            required=True,
        )  # type: ignore[assignment]

What is your recommendation to fix or ignore these?

Hey! I would like to run a cleanup function whenever the flow run is cancelled from the Cloud UI, but apparently using

Flow(..., on_failure=cleanup_fn)

does not trigger the function for cancellation. Oh and I'm using Prefect 1.0.

Hi, I'm running Prefect 1 and I'm a little lost on how to have a flow end if a given condition is met. For instance, I have a Flow performing an API call that returns rows updated in the last day, and if no rows are returned I'd like the flow to end right there. Typical python code like `exit`/`quit`/`pass` just cause the flow to "hang" in the cloud UI, but I'd like it to show a completed run.

Is it an accepted/intended practice to have a recurring flow read/write from Prefect v2 blocks? For example, I have a python app that interacts with the Azure Graph API... this api provides a "deltatoken" output that can be used as an input to the api for subsequent requests to capture net new changes. I've used dynamodb to cache this data up to this point but Im wondering if blocks could work for this as well.

Hi all, brand new user here. What's the 'right' way to execute an entire jupyter notebooks with prefect 2.0? I see the ExecuteNotebook class in Prefect 1.0, but nothing similar in 2.0. I can break up the notebooks into functions with task decorators, but my preference is to leave each notebooks 'as is' and run with a wrapper. What do folks recommend? Thanks!

Do you plan on bringing Artifacts to Prefect 2 ? (for example generating a link available in the UI after a flow run ?) I haven't find any alternative so far.

With prefect v1, we recently switched one of the flows to use

stored_as_as_script

when initializing storage. But when running the flow from the web UI, it now says

Failed to load and execute flow run: ValueError('No flows found in file.')

Hello Prefect folks! I'm searching for a pattern to manage dependencies of deployments. The 'hello world' way of calling a flow from another flow isn't a good fit as you will see by the components of my small data engineering pipeline: 1. Deployments A, B, C: DDL flow that creates tables on a DB. The flow is parametrized, the parameter is the filename of a DDL SQL script sent to a DB. So I have 3 deployments of same flow calling different scripts. 2. Deployment X: Data extracts that pull from tables built in deployments A, B and C. This is a single flow parametrized by the filename of several extract SQL scripts. I want to make sure Deployment X runs only after Deployments A, B, C have run each day. Currently I manage ordering by the cron schedule, however this isn't reliable if the agent isn't continuously running, or if a DDL job takes long time, etc. Can I have Deployment X explicitly wait on runs for other deployments (parametrized) run date?

Hi, I am trying to use Playwright python with prefect using docker. Though 'playwright' is installed, It gives me an error while running from prefect deployment : "ModuleNotFoundError: No module named 'playwright' ". But works fine while running python command.

hi 👋 When upgrading to

prefect 2.7.5

in our self hosted on kubernetes prefect I got the following error when starting a flow

Submission failed. kubernetes.client.exceptions.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': 'b79d317b-f68f-46ff-b226-09adb2d37b66', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': 'b89b22ad-0116-43a3-aa7b-0d04dc752da1', 'X-Kubernetes-Pf-Prioritylevel-Uid': '07475109-1722-457e-8c56-1612dc7046b5', 'Date': 'Wed, 04 Jan 2023 08:21:16 GMT', 'Content-Length': '349'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces \"kube-system\" is forbidden: User \"system:serviceaccount:prefect:prefect-conf-lfo\" cannot get resource \"namespaces\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"name":"kube-system","kind":"namespaces"},"code":403}

Nothing has changed in the cluster or service account and i I rollback to prefect

2.7.0

it works well Any idea / thoughts ?

I have this errors. Does any one know how to fix it 😭 thank you so much --------------------

from prefect import flow

@flow
def my_favorite_function():
    print("What is your favorite number?")
    return 42

print(my_favorite_function())

------------------------ ERROR:

Traceback (most recent call last):
  at block 1, line 8
  at /opt/python/envs/default/lib/python3.8/site-packages/prefect/flows.py, line 448, in __call__(self, return_state, wait_for, *args, **kwargs)
  at /opt/python/envs/default/lib/python3.8/site-packages/prefect/engine.py, line 164, in enter_flow_run_engine_from_flow_call(flow, parameters, wait_for, return_type)
  at /opt/python/envs/default/lib/python3.8/site-packages/anyio/_core/_eventloop.py, line 70, in run(func, backend, backend_options, *args)
  at /opt/python/envs/default/lib/python3.8/site-packages/anyio/_backends/_asyncio.py, line 292, in run(func, debug, use_uvloop, policy, *args)
  at /opt/python/lib/python3.8/asyncio/runners.py, line 33, in run(main, debug)
RuntimeError: asyncio.run() cannot be called from a running event loop

Hey folks - hopefully an easy question, just can't see it directly outlined anywhere. If I have a Deployment with default values in terms of how the Flow should be executed, does this mean that the execution will happen on the Agent?

Hey, I am using run_deployment to run multiple child flows from a parent flow. I want the parent flow to NOT wait a results of the child flows. Is this possible?

Hello, I am currently trying to set a prefect orion server in a docker on an EC2, however this error appears on the UI

Can't connect to Orion API at <http://0.0.0.0:80/api>. Check that it's accessible from your machine.

Can't find any help on internet besides doing

prefect config set PREFECT_API_URL=<http://0.0.0.0:80/api>

Any ideas ?

I have changed some things in my infrastructure block and now my flow runs crash with a 403 error:

Crash detected! Execution was interrupted by an unexpected exception: httpx.HTTPStatusError: Client error '403 Forbidden' for url '<https://api.prefect.cloud/api/accounts/uuid/workspaces/uuid/block_types/a8b20987-a71e-4b43-9350-53a9e0d7d467>'
For more information check: <https://httpstatuses.com/403>

uuid contains my respective identifiers. I am a bit confused, because I cant find any block with the id

a8b20987-a71e-4b43-9350-53a9e0d7d467

. Which is apparently what it want to access. Any tips?

Hello guys, I have a problem to connect to my existing dask cluster

my_flow.with_options(
        task_runner=DaskTaskRunner(
            address="<tls://10.4.83.2:8786>"
        )

Log:

prefect.task_runner.dask - Connecting to an existing Dask cluster at <tls://10.4.83.2:8786>
17:20:30.497 | ERROR   | Flow run 'jolly-griffin' - Crash detected! Execution was interrupted by an unexpected exception: TypeError: TLS expects a `ssl_context` argument of type ssl.SSLContext (perhaps check your TLS configuration?) Instead got None

What do I wrong? Dask cluster running on Kubernetes managed with Dask Gateway.

In Prefect Live at 3PM Eastern today on Twitch and YouTube, @Ryan Peden, the senior developer experience at Prefect, will talk about how to use cloud events to trigger Prefect automations. Add this event to your calendar so you don’t miss it.

Did anyone have this problem where the python client hangs on trying to set up deployment?

deployment = Deployment.build_from_flow(
    flow=load_data_from_minio,
    name="load_data_from_minio",
    work_queue_name="default",
    schedule=schedule,
    storage=storage_block,
    infrastructure=infra_block,
)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    deployment.apply()

Running this code just seems to hang forever. No debug logs either. Full code in thread

Hi there, I wanted to check my understanding of feature availability in Prefect Cloud 2.0. Is it possible to send, for specific flow state changes, custom Slack notifications? Ideally using the Slack block kit rather than plain text. Thanks!

Hi Team - We are trying to save a file in some sub-folders in the bucket we specified. However, the file is not saved in S3 sub-folders (specified in the base path), but in the bucket

csv_file = await s3_bucket_block.write_path(path=output_file, content=bytes_to_write)

Hi all, I just updated the prefect to 2.7.5 but all of my flow run state now becomes unknown.....how should I fix it?

Hi. I'm on Prefect 1. What is the simplest way to set a task to run as the terminal task of a flow? One could add requisite

upstream_tasks

but that's laborious when we want the behavior to be independent of the upstream, as long as it is serially the final task execution.