Hi, I'm having issues with prefect deployment build. It's taking a long time to build the deployment. Last time took me an hour

I am getting this isssue

assuming i’ve logged into prefect cloud, is there a way to temporaily disable logging a local run to cloud? an analogy is WandB, which supports a

WANDB_MODE=offline

, it would be great do be able to do something similar

Hi, we're now using prefect 2.0 personal plan and it seems so far so good. organization plan provides service accounts and read-only users ,but are they must-have features? because it will cost us around $7K annually.

Hi all! I’m trying to deploy my prefect 2.0 flows in parallel (in batches of 2), the deployments seem to to start at the same time, but it looks like the 2nd one is blocked until the first one completes. Here’s a snippet of what I’m trying to do in a python script:

for sub_deployments in flows:
    await asyncio.gather(
       *[deploy_flow(deployment_config) for deployment_config in sub_deployments]
    )

async def deploy_flow(deployment_config):

    flow = deployment_config.get("flow")

    print(f"deploying : {flow.name}")
    await Deployment.build_from_flow(
        ...
        storage=gcs_block,
        apply=True,
    )
    print(f"applied deployment: {flow.name}")

I get the two flows deploying at the same time, but the first one always finishes first, and I have to wait for bout 2 minutes until the 2nd one finishes and it’s always like that, which tells me that something’s blocking the 2nd flow from being deployed/registered with the API.

Hi y’all. Did something change with the Slack notifications? I’m getting these:

/my_path/.venv/lib/python3.10/site-packages/prefect/utilities/dispatch.py:162: UserWarning: Type 'SlackWebhook' at /my_path/.venv/lib/python3.10/site-packages/prefect_slack/credentials.py:43 has key 'slack-webhook' that matches existing registered type 'SlackWebhook' from /my_path/.venv/lib/python3.10/site-packages/prefect/blocks/notifications.py:83. The existing type will be overridden.

TypeError: Block.__new__() takes 1 positional argument but 2 were given

But the writing here still seems to be like how I’ve been using it (not using a Block). Maybe I missed another documentation update somewhere? Using: prefect 2.7.10 prefect-slack 0.1.1

is there a way to choose between a list of parameters that can be managed from the interface?

Hi everyone, it possible to use a task parameter as task name using

task.map

? I know the following works

@task
def say_hello(name: str) -> None:
    print(name)

names = ["a", "b"]
for name in names:
    say_hello.with_options(name=f"say {name}"}

But how do I make it work with mapping?

names = ["a", "b"]
say_hello.map(names)

I've asked the question a few days ago in an old thread (https://prefect-community.slack.com/archives/CL09KU1K7/p1674788888354029?thread_ts=1668092490.489519&cid=CL09KU1K7), asking it again here to get more attention

Is it possible to pass a custom name to a flow run in prefect 2.0?

Hi i am trying to trigger the flow run from the deployment api i.e. "http://orion:4200/api/deployments/" the body (JSON) i am sending

{
  "name": "deploymentUsingApi",
  "flow_id": "f27b41b3-6fd8-4356-b443-6cd08eab4cee",
  "is_schedule_active": true,
  "parameters": {},
  "tags": [
    "tag-1",
    "tag-2"
  ],
  "manifest_path": "api-manifest",
  "work_queue_name": "test_queue",
  "worker_pool_name": "my-worker-pool",
  "worker_pool_queue_name": "my-worker-pool-queue",
  "storage_document_id": "b2e8f87c-1c1f-499a-a143-a8f2b6253714",
  "infrastructure_document_id": "b2e8f87c-1c1f-499a-a143-a8f2b6253714",
  "schedule": {
    "interval": 0,
    "anchor_date": "2023-01-31T18:09:36.895Z",
    "timezone": "America/New_York"
  },
  "description": "creating deploymnent using api",
  "parameter_openapi_schema": {},
  "path": "",
  "version": "",
  "entrypoint": "flow.py:greetings",
  "infra_overrides": {}
}

The value assigned to storage_document_id and infrastructure_documnet_id is just a random uuid, since i trying this api with postman i m bit confused on entrypoint too that how to put it in correct way can anyone please help me on this.

Is there an easy way to mass delete late run flows? preferably on the cloud UI?

Is there a way to build deployments without having to install all dependencies? I’m trying to set up our CD pipeline for deploying our flows on ECS infrastructure and it looks like I have to install all the dependencies and set environment variables correctly to have

deployment build

work with an ecs-task infra block. This looks related but only looks like it pertains to the docker-container infra block. Otherwise we can manually create the deployments for now. We are following a similar pipeline to this one: https://github.com/anna-geller/dataflow-ops/blob/main/.github/workflows/ecs_prefect_agent.yml

Hello everyone! I implemented prefect open source in my company and we love it. I have a couple of questions though. We have pretty basic setup with k8s + RDS Postgres. Our flows have pretty big parameters (JSON with around 50k keys), so as you can imagine our flow_run parameters don’t fit in block size of postgres and so gets toasted. Not sure if toasting is the reason for the issue, but Prefect UI is loading like 20 seconds. I verified network connection between RDS & Orion (they are in same region), I checked EXAPLIN ANALYZE on query that orion executes to get flow_runs, and it blazing flast but when the query is executed it loads really long. Seem like client write issue, meaning client is taking so long to process received data from RDS. Did you encounter something like this? Or do you have any ideas how I can tackle this issue?

I am having an issue that is sporadic. I can't create a MRE, because it (seems to) occur randomly. I am more hoping that someone can point me at other discussion or github issues where this is discussed. I have been searching through discourse, slack and gh issues to find a possible solution. We will run a dozen or so flows at the same time. Sometimes one or two will be marked as failed because a number of tasks crashed with no error. Soemtimes a flow will be marked as crashed when I can see in GKE that it's still running? Has anyone encountered similar issues? prefect 2, gke, kubernetes jobs, using dask task runner. parent flow starts deployments and waits for them to show status. Is our agent potentially too small? 0.5vcpu and 2Gi mem.

Hello, Frank from Replica here. Yesterday we were having an issue with Flow Runs failing with `API_ERROR`s. Yesterday we were having similar issues, my colleague @Brett Naul reached out and the issue seemed resolved, however it's happening again. Can anyone advise? Details in thread. Thanks in advance.

Another question for the team: are there plans to allow for plugging in our own Notification plugins? (Orion open-source) And/or would there be interest in a PR for supporting Mattermost ?

Hi everyone , I am getting this error : crash detected but I am not able to source it down the reason

Executing 'OQCEA18L2-8a17dcaa-0' immediately...
08:33:57 AM
Crash detected! Execution was cancelled by the runtime environment.
09:00:19 AM
OQCEA18L2-8a17dcaa-0
Encountered exception during execution:
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.10/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
  File "/home/ubuntu/common_env/lib/python3.10/site-packages/prefect/engine.py", line 1614, in cancel_flow_run
    raise TerminationSignal(signal=signal.SIGTERM)
prefect.exceptions.TerminationSignal

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/common_env/lib/python3.10/site-packages/prefect/engine.py", line 643, in orchestrate_flow_run
    result = await run_sync(flow_call)
  File "/home/ubuntu/common_env/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 165, in run_sync_in_interruptible_worker_thread
    assert result is not NotSet
AssertionError

Can we clear flow runs in bulk ? IF yes how ?

Hello everyone, I have been trying to run a flow through an ECS Agent deployed in a fargate cluster. I am following this document for setting it up. But I keep getting the below error. I checked the all the roles and attached policies and all seems good. Does anybody know how to debug this?

Watching flow run execution...
└── 16:18:49 | INFO    | Entered state <Scheduled>: Flow run scheduled.
└── 16:18:57 | INFO    | Entered state <Submitted>: Submitted for execution
└── 16:18:57 | ERROR   | An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Role is not valid
└── 16:18:58 | INFO    | Entered state <Failed>: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Role is not valid
Flow run failed!

How can I login with

prefect cloud login

within CI if there are interactive results for login? Every time I try to login I am not met with exceptions due to having to type responses to various questions

Hi, how to run mapped tasks concurrently?

What is this warning and how do i make sure this is not affecting my runs ?

RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine';  in unpredictable behaviour

Hey all! Good morning from Portugal. Quick question, is it possible to limit the concurrency/parallelism on a mapped task?

Hi prefect community ! I try to run my flow on a kubernetes cluster that use VPC So actually i deployed my agent on that cluster, I can see that the flow get pulled but it can not create a pod on the cluster, because the prefect api requests are getting filtered. So my question is which ips should I whitelist to be able to run my flows on that cluster ?

A question on the ECS Deployment block: • Is it possible to have an agent run in one AWS account (centrally), but have that agent schedule ECS Task in other accounts, so other ECS clusters? This is all assuming of course that the trust relationship between the accounts exists. • Why? Because running 20+ agents in different accounts turns the whole setup from being nicely flexible (“serverless”) into a cost nightmare. We set up multiple accounts and were hoping that the central account would succeed in scheduling the tasks in the other accounts, but got straight-up submission errors in the UI:

Submission failed. Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/prefect_aws/ecs.py", line 797, in _create_task_and_wait_for_start task = self._run_task(ecs_client, task_run) File "/usr/local/lib/python3.9/site-packages/prefect_aws/ecs.py", line 1458, in _run_task return ecs_client.run_task(**task_run)["tasks"][0] File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 508, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 915, in _make_api_call raise error_class(parsed_response, operation_name) botocore.errorfactory.ClusterNotFoundException: An error occurred (ClusterNotFoundException) when calling the RunTask operation: Cluster not found. The above exception was the direct cause of the following exception: RuntimeError: Failed to run ECS task, cluster 'prefect' not found. Confirm that the cluster is configured in your region.

Edit Bonus Question: How would you debug this kind of issue? The main AWS exceptions seems to be suppressed unfortunately…

Hello! We're running Prefect version 1.4.1 in Kubernetes within AWS We want to be able to see the source/client IPs of requests made to Prefect services The Prefect UI pod's nginx.conf file seems to point to

/var/log/nginx/access.log

However, when I

tail

this file, it appears to be empty? Any idea how we can get these audit logs? Thanks!

I have a question about using subflows and passing large input parameters to subflows. Our team was hoping to write a standard subflow that happens after ingesting some data (mostly from various APIs) and resulting in a list output and the subflow would take the list output and take it on a standard process to get into our data warehouse. But when I try to pass a large amount of data to a subflow I am getting 403 Forbidden for url .... errors . Here is an example of what we had in mind would work.

from prefect import task, flow

@task(name="fake junk task")
def do_nothing(input):
    return input


@task(name="junk data task")
def make_junk_data(num_rows):
    return [
        {f"key_{i}": f"value_{i+j}" for i in range(500)}
        for j in range(num_rows)
    ]

@flow(name="subflow")
def subflow(data):
    do_nothing(data)

@flow(name="main flow")
def main_flow(num_rows):
    data = make_junk_data(num_rows)
    subflow(data=data)

if __name__ == "__main__":
    main_flow(50)

When main_flow input is 50 no problem. But if the input is increased to like 1000 it crashes and gives the error. httpx.HTTPStatusError: Client error '403 Forbidden' for url. By investigating the traceback it looks like the subflow is being "triggered" via API calls (to prefect cloud and with this huge input at a parameter to that call) in the background. So guessing this practice of passing a huge input as a parameter to a subflow is just not the ideal practice? Are there any recommendations for how to achieve this kind of pattern? I am thinking of writing the data first to file/cloud storage first and just pass the metadata for that instead but are there any parameters for subflows/tasks that would allow us to pass a huge list to a subflow like this? Thanks for any help!

Hi, when I navigate to the Prefect Orion UI Blocks -> Remote file System, and I have a S3 storage block, it shows credentials in the view:

{
  "use_ssl": false,
  "key": "real-key",
  "secret": "real-password",

Is it possible to hide the credentials somehow from the UI? Is there a planned feature to make the storage block view not to show the information? I have 2.7.8 Orion installed.

Hey guys! Can I apply/upload my prefect 2.0 deployments in parallel? When I tried, it seemed like prefect blocks the 2nd upload until the 1st one completes