Yash
01/21/2022, 12:08 PMdelete_flow_group_schedule
.
Is there any way to delete a 'programatically created schedule' by using graphql mutation or client through code.Lana Dann
01/21/2022, 6:43 PMJacob Longhurst
01/24/2022, 6:20 PMregistering agent
. I took a look through the code and it seems that on spin up the agent attempts a query in the db to see if it already exists. If it doesn’t then it registers itself by adding a new record of itself to the db. We are also not currently using the prefect cloud server but rather local instances of apollo, graphql, hasura, etc running in our k8s cluster
Here is the stack trace
[2022-01-24 18:15:41,310] INFO - agent | Registering agent...
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/prefect/cli/agent.py", line 331, in start
start_agent(KubernetesAgent, image_pull_secrets=image_pull_secrets, **kwargs)
File "/usr/local/lib/python3.6/site-packages/prefect/cli/agent.py", line 140, in start_agent
agent.start()
File "/usr/local/lib/python3.6/site-packages/prefect/agent/agent.py", line 189, in start
self._setup_api_connection()
File "/usr/local/lib/python3.6/site-packages/prefect/agent/agent.py", line 910, in _setup_api_connection
self.client.attach_headers({"X-PREFECT-AGENT-ID": self._register_agent()})
File "/usr/local/lib/python3.6/site-packages/prefect/agent/agent.py", line 858, in _register_agent
agent_config_id=self.agent_config_id,
File "/usr/local/lib/python3.6/site-packages/prefect/client/client.py", line 2107, in register_agent
agent_config_id=agent_config_id,
File "/usr/local/lib/python3.6/site-packages/prefect/client/client.py", line 570, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'message': '"\'Box\' object has no attribute \'data\'"', 'locations': [{'line': 2, 'column': 5}], 'path': ['register_agent'], 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': '"\'Box\' object has no attribute \'data\'"'}}}]
I can also provide the yaml for the k8sRomain
01/25/2022, 8:46 AMThe above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "flow_registration.py", line 136, in <module>
register_flows(args.local)
File "flow_registration.py", line 128, in register_flows
idempotency_key=flow.serialized_hash())
File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1734, in register
idempotency_key=idempotency_key,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 1213, in register
variables=dict(input=inputs),
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 555, in graphql
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 459, in post
retry_on_api_error=retry_on_api_error,
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 740, in _request
session=session, method=method, url=url, params=params, headers=headers
File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 661, in _send_request
raise ClientError(f"{exc}\n{graphql_msg}") from exc
prefect.exceptions.ClientError: 400 Client Error: Bad Request for url: <http://prefect-server-apollo.emtrails:4200/graphql>
The following error messages were provided by the GraphQL server:
GRAPHQL_VALIDATION_FAILED: Unknown type "register_tasks_input". Did you mean
"register_agent_input", "delete_agent_input", "delete_flow_input",
"delete_tenant_input", or "log_insert_input"?
GRAPHQL_VALIDATION_FAILED: Cannot query field "register_tasks" on type
"Mutation". Did you mean "register_agent", "delete_task", "insert_task", or
"update_task"?
The GraphQL query was:
mutation($input: register_tasks_input!) {
register_tasks(input: $input) {
success
}
}
Would anyone have encountered a similar issue?Alexander Melkoff
01/25/2022, 2:24 PMGITLAB_ACCESS_TOKEN
?Jen Evenson
01/25/2022, 6:35 PMjira.exceptions.JIRAError: JiraError HTTP 400 url: <https://our_jira_url/rest/api/2/issue>
text: expected Object containing a 'name' property
response text = {"errorMessages":[],"errors":{"assignee":"expected Object containing a 'name' property"}}
Lana Dann
01/25/2022, 7:06 PMAnders Segerberg
01/25/2022, 9:27 PMBen Welsh
01/26/2022, 12:17 PMJonas Bolin
01/26/2022, 12:37 PMdocker-compose up -d
would behave?Ben Welsh
01/26/2022, 2:09 PMCarlos Cueto
01/26/2022, 9:22 PMIan Armstrong
01/27/2022, 12:14 AMOops! It looks like something went wrong when trying to connect; make sure Prefect Server is running at the URL above and try again.
Any advice on how to debug this? GKE already creates a firewall rule to allow ingress over tcp:4200. Maybe something else weird with the firewall? I previously had Apollo set up as in ingress, but I switched back to a load balancer in case that would simplify things.Yash
01/27/2022, 11:04 AMScarlett King
01/27/2022, 12:15 PMBen Welsh
01/27/2022, 6:17 PMLiam England
01/27/2022, 7:30 PMBen Welsh
01/28/2022, 2:42 PMJamie Miller
01/28/2022, 3:01 PMTypeError: missing a required argument: 'target_date'
Jack Chang
01/28/2022, 6:59 PM[28 January 2022 10:46am]: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named '/Users/jackchang/'")
I went through some threads and added a gitlab repo as the storage and added that to my flow. Any ideas on what I should try next? 🙂Simon Hällqvist
01/29/2022, 10:04 AMPaulo Benatto
01/29/2022, 11:21 AMfrom prefect import Flow, task
import prefect
import time
from prefect.executors import LocalDaskExecutor
logger = prefect.context.get("logger")
@task
def add_ten(x):
time.sleep(1)
return x + 10
@task
def save_number(y):
<http://logger.info|logger.info>(f"Saving number: {y}")
with Flow("reduce", executor=LocalDaskExecutor()) as flow:
mapped_result = add_ten.map([1, 2, 3, 4, 5, 6, 7, 8, 9, 9])
mapped_result_2 = add_ten.map(mapped_result)
reduced_result = save_number(mapped_result_2)
Sanil Khurana
01/29/2022, 10:28 PMGagan Singh Saluja
01/30/2022, 8:17 PMBen Welsh
01/30/2022, 10:48 PMSylvain Hazard
01/31/2022, 1:15 PMBen Welsh
01/31/2022, 1:56 PMLocalDaskExecutor
to run mapped tasks in parallel inside a single GKE k8s container with what, if I understand the jargon correctly, is known as vertical scaling. Are there any resources that show the optimal way to do that?Bo
01/31/2022, 3:27 PMERROR - agent | Failed to infer default networkConfiguration, please explicitly configure using
--run-task-kwargs`` . However when I add the yaml path to the command, I start to receive this error: Error: No such option: --run-task-kwargs s3://*****/options.yaml
Any thoughts?Dotan Asselmann
01/31/2022, 5:35 PMTony Liberato
01/31/2022, 8:13 PMTony Liberato
01/31/2022, 8:13 PMKevin Kho
01/31/2022, 8:16 PMDocker()
? Maybe you can try adding Python dependencies as seen in the API to add the relevant Python packages?Tony Liberato
01/31/2022, 8:56 PMKevin Kho
01/31/2022, 9:05 PMTony Liberato
02/03/2022, 8:39 PMKevin Kho
02/03/2022, 8:40 PMTony Liberato
02/04/2022, 12:58 PM