Hello Everyone, I am very new to prefect and trying to run ECS container through ECS agent setup. I have provided the task definition arn, env variables, cluster name of ECS to the ECSRUN function and register the flow to the project. When I ran the flow, it gave the below error. Can anyone please explain me more about the error and how we can tackle that?

An error occurred (InvalidParameterException) when calling the RunTask operation: Override for container named flow is not a container in the TaskDefinition.

the machine I was using to host the prefect server failed over the weekend. When i restarted the machine and tried running

prefect server start --expose --use-volume

its not using the existing data/configurations i set up and in the logs im seeing the following messages:

hasura_1    | {"type":"startup","timestamp":"2022-03-21T19:50:54.856+0000","level":"error","detail":{"kind":"catalog_migrate","info":{"path":"$","error":"Cannot use database pr
eviously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47).","code":"not-supported"}}}
hasura_1    | {"path":"$","error":"Cannot use database previously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47)."
,"code":"not-supported"}

how do I go about debugging this?

Hello everyone! Is the

EXTRA_PIP_PACKAGES

environment variable supposed to work with

KubernetesFlowRunner

?

I'm in the process of trying to deploy prefect in a test environment and I'm at the point where the instructions in /getting-started tell me to run

prefect server create-tenant --name default

. I get that when deploying as a standalone service you need to create the tenant, unfortunately this command appears to be expecting prefect to be visible on localhost, which it is not. Is there some variable I was supposed to export to get the local CLI to see the remote prefect server?

Hey all 👋 we're bringing the 🍕 Pizza Patrol to Austin, TX! If you're going to Data Council or just live in the Lone Star State, we'd love to see you there. All are welcome for free pizza and drinks 🍺. More info on Meetup! https://prefect-community.slack.com/archives/C036FRC4KMW/p1647898467352239

Hello - in prefect 2.0, is there a way to provide the task name dynamically ? For example, if I have a function defined as a task which is being called multiple times from a flow, I want to append a dynamic suffix to the task name.

With Orion, do you think it would be feasible to have a task that returns a pre-configured fsspec filesystem that is set up from Orion Storage? We are using different storage types depending on the environment. I would like to be able to remove a lot of conditional code we have today with prefect 1.0, and it look like Orion with the Storage may be really helpful for that.

Hello. What is the correct way to fail task that have retry logic, but I am sure that even 100 retries will not help?

Hey all , Is it possible to trigger a prefect flow using the flow name currently running as prefect cloud server using AWS Lambda ??? Any suggestions ???

Hey all, 👋 We face several problems with flows that run in the Kubernetes cluster. • Pods often stuck in the

Running

state with the last message in logs

DEBUG - prefect.CloudFlowRunner | Checking flow run state...

• The flow in Prefect Cloud stucks in the

Cancelling

state and the pod stucks in the

Running

state in the Kubernetes cluster. Context: • prefect version

0.15.13

• Private Azure AKS cluster • We've tried to set

PREFECT__CLOUD__HEARTBEAT_MODE

to

"thread"

, but it only got worse (more stucked pods in the

Running

state). Now we have

PREFECT__CLOUD__HEARTBEAT_MODE

with

"process"

value and

tini -- prefect execute flow-run

as PID 1 to handle zombie process. It seems like the problem with the heartbeat process detecting the change to

Cancelling

or

Cancelled

states of the flow. I appreciate any help, thanks 🙂

Hey, i'm currently trying to do a simple ETL flow that connects to an API and fetch some records (i'm quite new to Prefect). I have to provide an API key to connect to this API as a query parameter. Is there some good practice regarding manipulation of this key ? Can i write it to another file that the flow will read ? i'm testing this locally but this will go in production env in the future (i hope)

I know that you can set retries on individual task failures. Can you set a retry on a "flow of flows" if one flow fails?

Hi everyone. I'd like to understand how memory is managed in a flow. I have a long-running flow that calls an API to get data. The flow works roughly like this: • get list of URLs to retrieve (about 180k URLs) • break the URLs in groups of 150 (list of lists) • a mapped task receives a list of 150 URLs and calls the API • another mapped task receives the API output for 150 URLs and saves the output to s3 I am using S3 results caching for the data intensive tasks (tasks 2 and 3 above) and Prefect Results for the rest of the tasks. I am seeing that the memory utilization keeps increasing until the container runs out of RAM (this is running on ECS fargate). It seems to be keeping the data retrieved from the API in memory, even after it's saved to s3. I can increase the container RAM but am trying to understand how I could write the flow so that it does not run out of RAM. This is what the Memory Utilization chart looks like. Eventually, the container dies and Prefect Cloud restarts it. Any suggestions?

Hello, any possibility of adding a -r flag or --run flag when one does

prefect deployment create deployment_name

that runs a deployment if a worker agent is already listening, if not, then its scheduled to run once upon creation of the worker queue.

What's the best way to set up a cronjob in a local machine with prefect code within a virtual environment. Currently did

pipenv shell prefect  agent local start

inside a sh script which I hands over to crontab like this

07 18 * * 1-5 bash startagent.sh

this works for sometime but suddenly stop saying it can't find or recognize those commands

Hi again, so I'm running a flow with KubernetesRun config in Prefect Cloud with a SqlServerFetch task that is receiving this error:

Error during execution of task: OperationalError('08001', '[08001] [Microsoft][ODBC Driver 17 for SQL Server]Client unable to establish connection because an error was encountered during handshakes before login. Common causes include client attempting to connect to an unsupported version of SQL Server, server too busy to accept new connections or a resource limitation (memory or maximum allowed connections) on the server. (26) (SQLDriverConnect)')

The task is defined first outside the scope of the flow and the task is mapped with the queries generated from another task. See the rough pattern below with some values obscured. The strange thing is that this pattern will run on my local machine and successfully pull from our database. Even stranger is that another flow uses SqlServerFetch with KubernetesRun config in Prefect Cloud successfully; though that job is mapped in the different way. Any thoughts on this? Only thing that comes to mind is some issue with the driver selected when mapping the task

Hi All, Any AWS + ECS experts here who have figured out how to monitor the CPU + memory utilization of

ECSRun

flows that execute on Fargate? I've noticed a couple of times that my ECS task will run out of memory and kill my flow, but when I go to the ECS (or Cloudwatch) console, there's no way for me to actually examine those metrics for the individual ECS task that was kicked off via my

ECSAgent

. From my digging around, it seems like it might have something to do with the fact that it's not actually the ECS service that my ECS task is tied to that's "responsible" for launching the ECS task (as is evidenced by the fact that the ECS dashboard says it's currently running 0 tasks even though the flow and the underlying ECS task are clearly running), but I'm not sure. I'm just trying to get better insight into the actual memory/CPU requirements of my flow, without having to, say, briefly move its execution to EC2, monitor it there, and then move it back to ECS Fargate... Many thanks in advance for any tips!

Yo all. Ran into an Orion issue and wondering if anyone had something similar. Basically I’m trying to create a flow + DeploymentSpec, then register it to Orion from within the same python file. (Currently have orion server running locally). @Darren was able to forward along this code snippet. But I’ve been getting an error getting it to work:

from prefect import flow
from prefect.deployments import DeploymentSpec, create_deployment_from_spec


@flow
def hello_world(name):
    print(f"Hello {name}!")


spec = DeploymentSpec(
    flow=hello_world,
    name="inline-deployment",
    parameters={"name": "Marvin"},
    tags=["foo", "bar"],
)

create_deployment_from_spec(spec=spec)

Meta question about discourse, is slack or discord the more active community for discussion?

When DaskExecutor is used, does Prefect delegate DAG (of tasks) scheduling all to DaskScheduler?

Hi. I have prefect server set up. I see the scheduled flows are submitted bu the agent never moves it to run state. Any idea why and what the fix could be?

Hi, Not sure whether I am doing something wrong or it is a bug but I am trying to run a mapped tasks that returns two values inside my flow an I am getting an error (code and trace inside)

Hi there, this might rather be an issue in form of a feature request, but in the HPC world, the preferred containerisation technology is Singularity rather than Docker. There are python libraries for handling Singularity in a similar way as Docker can be handled: https://github.com/singularityhub/singularity-cli. I could try to mimic the Docker task templates that are provided in prefect with singularity instead, would such a feature be useful for the community?

Hi, i'm just getting into logging. Is there more log type than info and warning ? if so where can i find this list ?

I am running a prefect agent with Orion right now with a deployed flow. The agent runs flows just fine if I start it and go hit quick run in the UI. But if I leave the agent sitting for too long I start getting this 403 error:

Traceback (most recent call last):
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/cli/base.py", line 59, in wrapper
    return fn(*args, **kwargs)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 120, in wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 67, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 56, in run
    return asynclib.run(func, *args, **backend_options)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 233, in run
    return native_run(wrapper(), debug=debug)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 228, in wrapper
    return await func(*args)
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/cli/agent.py", line 71, in start
    await agent.get_and_submit_flow_runs()
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/agent.py", line 88, in get_and_submit_flow_runs
    submittable_runs = await self.client.get_runs_in_work_queue(
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/client.py", line 747, in get_runs_in_work_queue
    response = await <http://self._client.post|self._client.post>(
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 137, in post
    return await self.request(
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 80, in request
    response.raise_for_status()
  File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/httpx/_models.py", line 1510, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '403 Forbidden' for url '<https://api-beta.prefect.io/api/accounts/df4b7089-cc2a-48ae-b4ce-baea44b163d6/workspaces/b22af91f-f810-4bc3-ac90-a1fa0e042c55/work_queues/c91e1439-be7e-4a98-8df0-da39515197b2/get_runs>'
For more information check: <https://httpstatuses.com/403>
An exception occurred.

Any ideas what might be causing this?

Hi guys , Is there a Query to execute or run a prefect flow from prefect Interactive API?

Hey all! Join @Kevin Kho and myself on PrefectLive 📺 today at 3p Eastern. He'll be running through some data validation workflows, and I'll be distracting Kevin. See 👀 you in chat!

Our team wrote a flow state handler that changes the new_state of the flow to Finished('message'). For some reason this causes our flows to look like they are running ad infinitum. For example The flows We ran a week ago show run duration of 7 days. But there isnt anything running as far as we can tell. Any ideas why changing the state to Finished State causes the flow runs in prefect UI to look like they are running forever? When I change our philosophy to setting the new state to Cancelled('message") we dont get see this behavior. The flow duration is exactly what it should be in that case. Just when we try to set the new state to a Finished instance.

Hello everyone, in prefect cloud is there any way to turn off the Lazarus retry? Thank you.

Hi all, I am experimenting with orion and I find it amazing so far. I noticed that there is no mention in the orion docs of logging the stdout, as is possible in prefect 1.0 . Also

@task(log_stdout=True)

gives an error in orion. Is there an alternative way to log the stdout in orion ?