Josh Rosenstein
06/22/2021, 3:42 PMfor flow in ${flows[@]}; do prefect register flow --file flows/$flow --project my_project --skip-if-flow-metadata-unchanged; done
(which currently works) to prefect register --path flows/ --project my_project
where my flows are utilizing azure storage and are located within flows folder. Using the new register command does create a new blob, but then errors out Error building storage: azure.core.exceptions.ResourceExistsError: Operation returned an invalid status 'The specified blob already exists.'
for each flow ultimately exiting out the job. For now I will just continue to use ``prefect register flow``, but wanted to see if anyone was having similar issues.Yueh Han Huang
06/22/2021, 5:39 PMBruno Murino
06/22/2021, 5:40 PM[2021-06-22 13:25:46+0100] INFO - prefect.ShellTask | 13:25:46 | Concurrency: 12 threads (target='dev')
INFO:prefect.ShellTask:13:25:46 | Concurrency: 12 threads (target='dev')
Bruno Murino
06/22/2021, 5:41 PMstream_output=True
on the shelltaskHugo Kitano
06/22/2021, 5:49 PMfrom prefect import Client
client = Client()
client.set_secret(name="MYSECRET", value="MY SECRET VALUE")
I can see ‘MYSECRET’ on my cloud UI, so I’m sure I’m authorized etc. But I am unable to get a secret with:
from prefect.client import Secret
# Load the value of `MYSECRET`
my_secret_value = Secret("MYSECRET").get()
Where I get a *** ValueError: Local Secret "MYSECRET" was not found.
error.
Reading the docs, I see that it should be looking for a local secret first, and if not found, will look for a cloud secret. I think it is not looking for a cloud secretCharles Liu
06/22/2021, 6:11 PMprefect agent kubernetes start --job-template
again to amend my existing agent? I'm trying to edit the safe-to-evict setting.Tim Enders
06/22/2021, 6:25 PMTim Enders
06/22/2021, 6:29 PMcontext
exists, but I am looking for something that can read a file and set the state. I don't think the context object is setup for that. It seems to be more runtime to meCarter Kwon
06/22/2021, 8:29 PMScott Vermillion
06/22/2021, 8:45 PMLuis Henrique
06/22/2021, 9:00 PMJustin Liu
06/22/2021, 9:15 PMHayden Sansum
06/22/2021, 9:53 PMmatta
06/22/2021, 10:59 PMTrine Bruun Brockhoff
06/23/2021, 9:44 AMMichael Law
06/23/2021, 10:02 AM[2021-06-23 08:53:35,179] ERROR - agent | Error while managing existing k8s jobs
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 384, in heartbeat
self.manage_jobs()
File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 230, in manage_jobs
pod_events.items, key=lambda x: x.last_timestamp
TypeError: '<' not supported between instances of 'datetime.datetime' and 'NoneType'
ERROR:agent:Error while managing existing k8s jobs
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 384, in heartbeat
self.manage_jobs()
File "/usr/local/lib/python3.7/site-packages/prefect/agent/kubernetes/agent.py", line 230, in manage_jobs
pod_events.items, key=lambda x: x.last_timestamp
TypeError: '<' not supported between instances of 'datetime.datetime' and 'NoneType'
Adam
06/23/2021, 10:13 AMERROR - prefect.CloudFlowRunner | Unexpected error: AttributeError("'Log' object has no attribute 'splitlines'")
I’m running prefect self hosted. Server, agent & flow versions are 0.14.22
Nothing has changed, any idea what could be the reason?Marwan Sarieddine
06/23/2021, 12:45 PMKevin Kho
Sean Perry
06/23/2021, 5:01 PMFlorian Kühnlenz
06/23/2021, 6:09 PMLeon Kozlowski
06/23/2021, 6:33 PMBen Muller
06/23/2021, 8:07 PMmatta
06/23/2021, 10:06 PMconfig.toml
with my auth token (stored as a secret in GitHub)
4. run prefect register
through the CLI
Thanks!Ben Muller
06/23/2021, 10:43 PMDocker
storage to push an image to ECR.
I am authenticated with aws sts assume-role
locally in my CLI.
I am trying to push the image locally at the moment and getting an error with:
File "/Users/benmuller/code/es-betfair-flows/.venv/lib/python3.8/site-packages/prefect/storage/docker.py", line 585, in push_image
raise InterruptedError(line.get("error"))
InterruptedError: denied: Your authorization token has expired. Reauthenticate and try again.
How can I make sure that my sts
creds are honoured ?Aiden Price
06/23/2021, 11:20 PMFina Silva-Santisteban
06/23/2021, 11:27 PMpip
and python 2.7
error? (pls see screenshots) The dockerfile I’m using for the docker storage (flow.storage = Docker(dockerfile="Dockerfile", …)
looks like this
FROM python:3.9
WORKDIR /app
COPY . .
ENV PYTHONPATH=$PYTHONPATH:`pwd`
FROM <http://gcr.io/google-appengine/python|gcr.io/google-appengine/python>
RUN apt-get update
RUN apt-get install poppler-utils -y
I’ve set the python version to 3.9 in the dockerfile, why doesn’t it use that?Ben Muller
06/24/2021, 5:01 AMprefect.context.get("logger")
to log throught our entire project ? eg. when sub modules are called by calling their classes in that flow?Varuna Bamunusinghe
06/24/2021, 5:36 AMJoël Luijmes
06/24/2021, 7:26 AM