Mohan kancherla
03/21/2022, 7:56 PMAn error occurred (InvalidParameterException) when calling the RunTask operation: Override for container named flow is not a container in the TaskDefinition.
joshua mclellan
03/21/2022, 7:58 PMprefect server start --expose --use-volume
its not using the existing data/configurations i set up and in the logs im seeing the following messages:
hasura_1 | {"type":"startup","timestamp":"2022-03-21T19:50:54.856+0000","level":"error","detail":{"kind":"catalog_migrate","info":{"path":"$","error":"Cannot use database pr
eviously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47).","code":"not-supported"}}}
hasura_1 | {"path":"$","error":"Cannot use database previously used with a newer version of graphql-engine (expected a catalog version <=40, but the current version is 47)."
,"code":"not-supported"}
how do I go about debugging this?Anatoly Myachev
03/21/2022, 8:33 PMEXTRA_PIP_PACKAGES
environment variable supposed to work with KubernetesFlowRunner
?Michael Aldridge
03/21/2022, 9:08 PMprefect server create-tenant --name default
. I get that when deploying as a standalone service you need to create the tenant, unfortunately this command appears to be expecting prefect to be visible on localhost, which it is not. Is there some variable I was supposed to export to get the local CLI to see the remote prefect server?Chris Reuter
03/21/2022, 9:35 PMDarshan
03/21/2022, 11:41 PMdavzucky
03/22/2022, 12:46 AMVadym Dytyniak
03/22/2022, 8:53 AMShrikkanth
03/22/2022, 10:59 AMandrr
03/22/2022, 12:50 PMRunning
state with the last message in logs DEBUG - prefect.CloudFlowRunner | Checking flow run state...
• The flow in Prefect Cloud stucks in the Cancelling
state and the pod stucks in the Running
state in the Kubernetes cluster.
Context:
• prefect version 0.15.13
• Private Azure AKS cluster
• We've tried to set PREFECT__CLOUD__HEARTBEAT_MODE
to "thread"
, but it only got worse (more stucked pods in the Running
state). Now we have PREFECT__CLOUD__HEARTBEAT_MODE
with "process"
value and tini -- prefect execute flow-run
as PID 1 to handle zombie process.
It seems like the problem with the heartbeat process detecting the change to Cancelling
or Cancelled
states of the flow.
I appreciate any help, thanks 🙂Florian Guily
03/22/2022, 1:13 PMJason Motley
03/22/2022, 2:30 PMPedro Machado
03/22/2022, 4:43 PMRajan Subramanian
03/22/2022, 5:55 PMprefect deployment create deployment_name
that runs a deployment if a worker agent is already listening, if not, then its scheduled to run once upon creation of the worker queue.Hedgar
03/22/2022, 6:32 PMpipenv shell prefect agent local start
inside a sh script which I hands over to crontab like this 07 18 * * 1-5 bash startagent.sh
this works for sometime but suddenly stop saying it can't find or recognize those commandsDavid Beck
03/22/2022, 7:37 PMError during execution of task: OperationalError('08001', '[08001] [Microsoft][ODBC Driver 17 for SQL Server]Client unable to establish connection because an error was encountered during handshakes before login. Common causes include client attempting to connect to an unsupported version of SQL Server, server too busy to accept new connections or a resource limitation (memory or maximum allowed connections) on the server. (26) (SQLDriverConnect)')
The task is defined first outside the scope of the flow and the task is mapped with the queries generated from another task. See the rough pattern below with some values obscured.
The strange thing is that this pattern will run on my local machine and successfully pull from our database. Even stranger is that another flow uses SqlServerFetch with KubernetesRun config in Prefect Cloud successfully; though that job is mapped in the different way. Any thoughts on this? Only thing that comes to mind is some issue with the driver selected when mapping the taskSean Talia
03/22/2022, 7:47 PMECSRun
flows that execute on Fargate? I've noticed a couple of times that my ECS task will run out of memory and kill my flow, but when I go to the ECS (or Cloudwatch) console, there's no way for me to actually examine those metrics for the individual ECS task that was kicked off via my ECSAgent
. From my digging around, it seems like it might have something to do with the fact that it's not actually the ECS service that my ECS task is tied to that's "responsible" for launching the ECS task (as is evidenced by the fact that the ECS dashboard says it's currently running 0 tasks even though the flow and the underlying ECS task are clearly running), but I'm not sure. I'm just trying to get better insight into the actual memory/CPU requirements of my flow, without having to, say, briefly move its execution to EC2, monitor it there, and then move it back to ECS Fargate...
Many thanks in advance for any tips!Daniel Chapsky
03/22/2022, 9:32 PMfrom prefect import flow
from prefect.deployments import DeploymentSpec, create_deployment_from_spec
@flow
def hello_world(name):
print(f"Hello {name}!")
spec = DeploymentSpec(
flow=hello_world,
name="inline-deployment",
parameters={"name": "Marvin"},
tags=["foo", "bar"],
)
create_deployment_from_spec(spec=spec)
Shiyu Gan
03/23/2022, 2:51 AMShiyu Gan
03/23/2022, 3:24 AMArchitha Rao
03/23/2022, 6:41 AMTomer Cagan
03/23/2022, 7:31 AMPaul Gierz
03/23/2022, 8:59 AMFlorian Guily
03/23/2022, 10:39 AMNelson Griffiths
03/23/2022, 1:20 PMTraceback (most recent call last):
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/cli/base.py", line 59, in wrapper
return fn(*args, **kwargs)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 120, in wrapper
return run_async_in_new_loop(async_fn, *args, **kwargs)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 67, in run_async_in_new_loop
return anyio.run(partial(__fn, *args, **kwargs))
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 56, in run
return asynclib.run(func, *args, **backend_options)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 233, in run
return native_run(wrapper(), debug=debug)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 228, in wrapper
return await func(*args)
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/cli/agent.py", line 71, in start
await agent.get_and_submit_flow_runs()
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/agent.py", line 88, in get_and_submit_flow_runs
submittable_runs = await self.client.get_runs_in_work_queue(
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/client.py", line 747, in get_runs_in_work_queue
response = await <http://self._client.post|self._client.post>(
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 137, in post
return await self.request(
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 80, in request
response.raise_for_status()
File "/home/nelson/miniconda3/envs/my_project/lib/python3.9/site-packages/httpx/_models.py", line 1510, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '403 Forbidden' for url '<https://api-beta.prefect.io/api/accounts/df4b7089-cc2a-48ae-b4ce-baea44b163d6/workspaces/b22af91f-f810-4bc3-ac90-a1fa0e042c55/work_queues/c91e1439-be7e-4a98-8df0-da39515197b2/get_runs>'
For more information check: <https://httpstatuses.com/403>
An exception occurred.
Any ideas what might be causing this?Shrikkanth
03/23/2022, 2:16 PMChris Reuter
03/23/2022, 2:33 PMKyle Austin
03/23/2022, 2:34 PMDaniel Komisar
03/23/2022, 3:44 PMDomenico Di Gangi
03/23/2022, 3:44 PM@task(log_stdout=True)
gives an error in orion. Is there an alternative way to log the stdout in orion ?