Dana Merrick
03/26/2021, 9:18 PMRunNamespacedJob
task?ReadNamespacedPodLogs
and i found the PR that added it but I dont understand how to use itnicholas
03/26/2021, 9:20 PMlog_level
to your task to get logs from the job. (by default this is set to None
, which disables logging from the job)Dana Merrick
03/26/2021, 9:20 PM"debug"
for the log level but still get no outputnicholas
03/26/2021, 9:26 PMlog_level='debug'
should work, can you send your task declaration?Dana Merrick
03/26/2021, 9:26 PMflow = s3_flow(sys.argv[0])
flow.add_task(run_command("ls -al"))
if __name__ == "__main__":
flow.run()
def run_command(command, image=None):
# create a length-6 random string
name = "".join(random.choices(string.ascii_uppercase + string.digits, k=6))
image = image or DEFAULT_IMAGE
body = {
"apiVersion": "batch/v1",
"kind": "Job",
"metadata": {"name": name},
"spec": {
"template": {"spec": {"containers": [], "restartPolicy": "Never"}},
"backoffLimit": 4,
},
}
container = {"name": "executor", "image": image, "command": command.split(" ")}
body["spec"]["template"]["spec"]["containers"].insert(0, container)
return RunNamespacedJob(
body=body,
delete_job_after_completion=True,
kubernetes_api_key_secret=False,
log_level="debug",
log_stdout=True,
namespace="default",
)
nicholas
03/26/2021, 9:29 PM.run()
on itDana Merrick
03/26/2021, 9:30 PMs3_flow()
, it's:def s3_flow(flow_name, project_name=None, image=None):
docker_image = image or DEFAULT_IMAGE
project_name = project_name or DEFAULT_PROJECT
flow = Flow(
flow_name,
run_config=KubernetesRun(labels=DEFAULT_LABELS, image=docker_image),
storage=S3(bucket=DEFAULT_BUCKET),
)
return flow
run_command()
does get run, it just gets swallowednicholas
03/26/2021, 9:35 PMrun_command
is a task, its run method is being called by the flow runner; however since it's also returning a task (your, the runner doesn't know to call the run
method; try this:
return RunNamespacedJob(
body=body,
delete_job_after_completion=True,
kubernetes_api_key_secret=False,
log_level="debug",
log_stdout=True,
namespace="default",
).run()
Dana Merrick
03/26/2021, 9:35 PM17:50:07
INFO
CloudTaskRunner
Task 'run_command': Starting task run...
17:50:07
INFO
RunNamespacedJob
Job 4au0dr has been created.
17:52:18
INFO
RunNamespacedJob
Started following logs for 4au0dr-ssc84
17:52:18
INFO
RunNamespacedJob
Job 4au0dr has been completed.
17:52:18
INFO
CloudTaskRunner
Task 'run_command': Finished task run for task with final state: 'Success'
total 140
drwxr-xr-x 1 root root 44 Mar 26 21:48 .
drwxr-xr-x 1 root root 17 Mar 26 21:52 ..
-rw-r--r-- 1 root root 216 Mar 26 21:48 .dockerignore
drwxr-xr-x 2 root root 6 Mar 26 16:07 .empty
-rw-r--r-- 1 root root 65 Mar 26 21:48 .gitattributes
-rw-r--r-- 1 root root 257 Mar 26 21:48 .gitignore
...
nicholas
03/26/2021, 10:36 PMlog_level="info"
to your task constructor instead of debug
If your norma log level is set atDana Merrick
03/26/2021, 10:45 PM@task()
the task constructor?RunNamespacedJob()
nicholas
03/26/2021, 10:49 PMRunNamespacedJob()
, as you were before but with a different log levelDana Merrick
03/26/2021, 10:49 PMJoël Luijmes
03/27/2021, 6:43 PMpods/log
If you use prrefect-server this recently was included in the helm chart, if not the role should look something like
rules:
- apiGroups:
- batch
- extensions
resources:
- jobs
- jobs/status
verbs:
- '*'
- apiGroups:
- ""
resources:
- events
- pods
- pods/log
- services
verbs:
- '*'
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- '*'
Dana Merrick
03/28/2021, 4:04 PMJoël Luijmes
04/06/2021, 9:49 AM