Jonathan Mathews
05/05/2022, 5:57 PMPatrick Tan
05/05/2022, 6:09 PMJonathan Mathews
05/05/2022, 9:14 PMLeon Kozlowski
05/05/2022, 9:32 PMTraceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/prefect/agent/kubernetes/agent.py", line 413, in heartbeat
self.manage_jobs()
File "/usr/local/lib/python3.8/site-packages/prefect/agent/kubernetes/agent.py", line 193, in manage_jobs
f"Job {job.name!r} is for flow run {flow_run_id!r} "
AttributeError: 'V1Job' object has no attribute 'name'
I found a thread discussing this - but I wanted to know if there was another way to alleviate this other than upgrading to prefect 1.0+ - If that is the only solution, are there any gotchas/things I should look out for when upgrading? And will flows as old as 0.14.X still run as expected?Seth Just
05/05/2022, 10:09 PMLeon Kozlowski
05/06/2022, 12:18 AMAlex Shea
05/06/2022, 12:36 AMArtem Vysotsky
05/06/2022, 1:02 AMGustavo Frederico
05/06/2022, 1:39 AMGustavo Frederico
05/06/2022, 1:55 AMEddie Atkinson
05/06/2022, 2:48 AMFargateCluster
. It seems like the cluster is being assigned a public IP address. Ideally I wouldn’t want that to be the case as I don’t want people snooping on my cluster / submitting jobs. However, when I pass "fargate_private_ip": True
to cluster_kwargs
my cluster fails to start with the error: Cluster failed to start: Timed out trying to connect to <tcp://10.0.1.111:8786> after 30 s
That makes sense. Someone somewhere failed to connect to a local IP address, presumably from outside the subnet. What I don’t understand is how I can prevent people from arbitrarily accessing my cluster from the internet whilst allowing all the ‘right’ traffic throughJeff Kehler
05/06/2022, 6:40 AMIntervalClock
scheduler. If i were to pass interval=timedelta(hours=1)
to schedule a run for every hour, would this be scheduled to occur at the beginning of every hour? Or would it be relative to when I register the Flow? eg if I register the flow at 1:13pm then the IntervalClock
would trigger at 13 minutes past the hour every hour? Or would it just kick off at the first minute of every hour?Thomas Huijskens
05/06/2022, 7:57 AMIevgenii Martynenko
05/06/2022, 8:16 AMJonathan Mathews
05/06/2022, 8:43 AMIevgenii Martynenko
05/06/2022, 12:20 PMtoken = EnvVarSecret("TOKEN")
connections = get_connections(token=token) # this is a task that returns dict(name, connection_string)
start_task_result = start_task(connections.get('some_key')) # i'd expect to get value from dict, not attribute error
Nikhil Joseph
05/06/2022, 12:54 PMJovan Sakovic
05/06/2022, 1:06 PM--watch
flag.
However, as we’re spinning it all up with docker-compose, and we have multiple Prefect Projects, so I want to run a few of these prefect register
commands in the container’s entrypoint shell script (code example in 🧵)
Is there an easy way to:
• either run the prefect register
command in background, so it lets the rest run
• or, run prefect register
with specifying the project in the Flow script, so we’d need to run the command only once for all projectsMia
05/06/2022, 1:30 PMHenning Holgersen
05/06/2022, 2:05 PMAlvaro Durán Tovar
05/06/2022, 2:16 PMJai P
05/06/2022, 2:25 PMLeon Kozlowski
05/06/2022, 4:27 PMEvan Curtin
05/06/2022, 6:12 PMZach Schumacher
05/06/2022, 8:03 PMCloudFlowRunner
so that we can initialize sentry and datadog as early as possible, as well as flush our statsd buffer to track metrics.
it seems like the place i'd expect it to read from the env var is hardcoded tho https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/kubernetes/agent.py#L562-L586Mars
05/06/2022, 8:03 PMXavier Babu
05/06/2022, 8:04 PMDarin Douglass
05/06/2022, 8:27 PMDavid Haynes
05/06/2022, 9:07 PMKathryn Klarich
05/06/2022, 9:46 PMValueError: You have not set an API key for authentication.
Is anyone familiar with this error? Was something updated in prefect that could have caused this crash?