Josh Paulin
06/17/2022, 2:42 PMPREFECT__CLOUD__USE_LOCAL_SECRETS
and PREFECT__CONTEXT__SECRETS__GITLAB_ACCESS_TOKEN
set on my agent, but not on the job. Trying to run the flow just errors out at
Failed to load and execute flow run: KeyError('The secret GITLAB_ACCESS_TOKEN was not found. Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>')
Kevin Kho
flow.run()
or an agent run?Josh Paulin
06/17/2022, 2:45 PMERROR: execute flow-run
in the web UI.Kevin Kho
Josh Paulin
06/17/2022, 2:48 PMcontainers:
- args:
- prefect agent kubernetes start
command:
- /bin/bash
- -c
env:
- name: PREFECT__CLOUD__AGENT__AUTH_TOKEN
valueFrom:
secretKeyRef:
name: prefect-secrets
key: prefect-key
...
- name: PREFECT__CLOUD__AGENT__AGENT_ADDRESS
value: http://:8080
- name: PREFECT__CLOUD__API_KEY
valueFrom:
secretKeyRef:
name: prefect-secrets
key: prefect-key
- name: PREFECT__CLOUD__TENANT_ID
value: ''
- name: PREFECT__CLOUD__USE_LOCAL_SECRETS
value: 'true'
- name: PREFECT__CONTEXT__SECRETS__GITLAB_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: prefect-secrets
key: prefect-gitlab-token
Kevin Kho
prefect agent kubernetes install --env TEST=false
and you still see the format for including env variables that are passed through to the flowJosh Paulin
06/17/2022, 2:55 PMPREFECT__CLOUD__AGENT__ENV_VARS
?Kevin Kho
Josh Paulin
06/17/2022, 2:57 PMKevin Kho
Josh Paulin
06/17/2022, 3:02 PMKevin Kho
Josh Paulin
06/17/2022, 3:08 PM[Errno 2] No such file or directory: 'job-template.yml'
).
The flow code for refference
@task
def hello_task():
logger = prefect.context.get('logger')
<http://logger.info|logger.info>('Hello world!')
with Flow('hello-flow') as flow:
hello_task()
flow.storage = GitLab(
host='<HOST>',
repo='<REPO>',
path='flows/hello_flow.py',
ref='<BRANCH>'
)
flow.run_config = KubernetesRun(job_template_path='job-template.yml')
flow.register(project_name='tutorial')
Kevin Kho
s3://
. Point being your agent needs access to load the file during run timeJosh Paulin
06/17/2022, 6:44 PMKevin Kho
Josh Paulin
06/17/2022, 6:46 PMKevin Kho
Josh Paulin
06/17/2022, 6:49 PMKevin Kho
job_template_path (str, optional): Path to a job template to use. If a local path (no file scheme, or a file/local scheme), the job template will be loaded on initialization and stored on the KubernetesRun object as the job_template field. Otherwise the job template will be loaded at runtime on the agent. Supported runtime file schemes include (s3, gcs, and agent (for paths local to the runtime agent)).
so I guess it should have been read already when you registered?Josh Paulin
06/17/2022, 6:56 PM[Errno 2] No such file or directory: 'job-template.yml'
Traceback (most recent call last):
File "/usr/local/bin/prefect", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/prefect/cli/execute.py", line 96, in flow_run
raise exc
File "/usr/local/lib/python3.8/site-packages/prefect/cli/execute.py", line 73, in flow_run
flow = storage.get_flow(flow_data.name)
File "/usr/local/lib/python3.8/site-packages/prefect/storage/gitlab.py", line 105, in get_flow
return extract_flow_from_file(
File "/usr/local/lib/python3.8/site-packages/prefect/utilities/storage.py", line 88, in extract_flow_from_file
exec(contents, exec_vals)
File "<string>", line 24, in <module>
File "/usr/local/lib/python3.8/site-packages/prefect/run_configs/kubernetes.py", line 106, in __init__
with open(parsed.path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'job-template.yml'
Kevin Kho
from prefect.utilities.filesystems import parse_path
parsed = parse_path("job-template.yaml")
print(parsed.scheme)
Josh Paulin
06/17/2022, 7:20 PMfile
Kevin Kho
from prefect.run_configs import KubernetesRun
a = KubernetesRun(job_template_path="test-yan.yaml")
print(a.job_template)
this is working for me. I am guessing the Gitlab script based storage just re-evaluates the file to obtain the Flow, and as a side effect of that, it tries to instantiate the KubernetesRun even though it’s already been read, you know what I mean?Josh Paulin
06/17/2022, 7:31 PMKevin Kho
flow
variable. Yes the Git repo gets cloned, but it’s in a temp directory and doesn’t find these files. Maybe we can use Git
storage instead of Gitlab
because Git
storage has a way to load in these files like shown here, but then the path needs to be changed. Let me think about this.Josh Paulin
06/17/2022, 8:21 PMKevin Kho
with Flow(...) as flow:
...
...
if __name__ == "__main__":
flow.run_config = KubernetesRun(...)
flow.register(...)
this way, it will only surely run during registrationJosh Paulin
06/17/2022, 8:24 PMKevin Kho