Aqib Fayyaz
10/21/2021, 9:01 PMprefect agent kubernetes install -k API_KEY | kubectl apply --namespace=my-namespace -f -
this command successfully creates the deployment and referrs to default prefect image so the question is how can i use my code with it?Tony Yun
10/21/2021, 9:04 PMhaf
10/21/2021, 9:58 PMFailed to retrieve task state with error: ClientError([{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
task_run_info = self.client.get_task_run_info(
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 1798, in get_task_run_info
result = self.graphql(mutation) # type: Any
File "/usr/local/lib/python3.8/site-packages/prefect/client/client.py", line 569, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}]
William Grim
10/21/2021, 10:36 PMprefect get logs --id uuid_of_that_flow
, it gets killed with no logs retrieved.
Is there another place to go look for logs?Tony Yun
10/21/2021, 10:46 PMQueued
with just one that task running. Only if I increase the limit to 2 it will start running. It looks a bug to me:Maria
10/21/2021, 11:05 PMAn error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
and I'm aware there is open github issue (https://github.com/PrefectHQ/prefect/issues/4402). We did adjust params as recommended but it still happens. I want to be able to restart the flow but cannot use task retry feature as no tasks have been started (it is the only log line I see and there is no start time for the flow ):Adam Everington
10/22/2021, 7:57 AMEddie Atkinson
10/22/2021, 9:12 AMgraphql-code-generator
but that hasn’t worked so farhaf
10/22/2021, 10:50 AMAdrien B
10/22/2021, 12:32 PMToprak Nihat Deniz Öztürk
10/22/2021, 1:50 PMf_run = flow.run
method and access the results with f_run.results[df].result
.Thomas Furmston
10/22/2021, 3:04 PMJoão Luiz Carabetta
10/22/2021, 3:57 PMstate = my_favorite_function()
Sahil Chopra
10/22/2021, 5:15 PMEmailTask
what is the EMAIL_USERNAME
and EMAIL_PASSWORD
used for? Are the notifications shipped from this email? Is this required, or is it possible to use the default notifications@prefect.io email address shown in the class definition?Hugo Kitano
10/22/2021, 5:28 PMquery {
flow_run(where: {name: {_eq: "woodoo-leopard"}}) {
id
state
start_time
}
}
Erik Nilsson
10/22/2021, 5:55 PMGiovanni Giacco
10/22/2021, 6:03 PMCharles Liu
10/22/2021, 6:29 PMJacob Goldberg
10/22/2021, 6:53 PMValueError: Local Secret "SLACK_WEBHOOK_URL" was not found.
Tony Yun
10/22/2021, 8:01 PMAbhas P
10/22/2021, 8:09 PMJoe Still
10/22/2021, 9:04 PMMatt Alhonte
10/22/2021, 11:17 PMif notebook_name in list_of_notebooks:
run_notebook(notebook_name)
else:
print("skipped")
How do I do that with the StartFlowRun
task? Should I subclass it and replace the existing .run()
method?Huw Ringer
10/23/2021, 11:01 PMmap
method: process_tasks.map(flow_description, start_task_id)
My problem is that if one of the mapped tasks fails, the other ones still subsequently run regardless, and I’d like for the whole flow to be cancelled programatically/automatically when that happens. I think the CancelFlowRun API method should be able to do this if I invoke it just before the failed task enters the failed state (via a raise exception). I’m struggling to figure out how to call this thing though, and was hoping someone might be able to help, please. If I try the following:
prefect.tasks.prefect.flow_run_cancel.CancelFlowRun.run(context.get("flow_run_id"))
My python engine says it can’t find the flow_run_cancel
method in any imported Prefect module, and I’ve already imported task, Flow, Parameter, context, Task, tasks
(for good measure). I’m clearly doing something very wrong and don’t know enough to figure out what it is. Would be very grateful for any help or suggestions to put me out of my misery. Thanks in advance!haf
10/24/2021, 7:37 AMGabi Pi
10/24/2021, 12:17 PMRunNamespacedJob
task over Kubernetes. I am using the following code:
storage = S3(bucket="gavriel-test", stored_as_script=True)
kubernetes_run_conf = KubernetesRun(
env={
"AWS_ACCESS_KEY_ID": AWS_ACCESS_KEY_ID,
"AWS_SECRET_ACCESS_KEY": AWS_SECRET_ACCESS_KEY
},
labels=["prefect-poc-k8s-agent"]
)
body = {
'apiVersion': 'batch/v1',
'kind': 'Job',
'metadata': {'name': 'echo'},
'spec':
{
'template':
{
'spec': {
'containers': [
{
'name': 'echo',
'image': 'alpine:3.7',
'command': ['sh -c "echo Hello!!!"; sleep 10']
}
]
}
},
'backoffLimit': 4
}
}
with Flow("kubernetes-CreateNamespacedJob", run_config=kubernetes_run_conf, storage=storage) as flow:
job = RunNamespacedJob(body=body, namespace="prefect", delete_job_after_completion=False)
But I keep getting the following error:
Error during execution of task: MaxRetryError("HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /apis/batch/v1/namespaces/prefect/jobs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8c8adeb650>: Failed to establish a new connection: [Errno 111] Connection refused'))")
Do you have any idea how to resolve this?Royzac
10/24/2021, 11:08 PMTony Yun
10/25/2021, 3:09 AM