Adam Brusselback
10/06/2021, 3:24 AMAntti Tupamäki
10/06/2021, 6:27 AMJelle Vegter
10/06/2021, 7:41 AMWill
10/06/2021, 11:01 AMAndreas Tsangarides
10/06/2021, 12:04 PMstored_as_script
argument in Docker() along with path
, but cannot see how I can avoid building and pushing an image for each flow that way...Lucas Giger
10/06/2021, 1:12 PMBob Colner
10/06/2021, 2:25 PMBob Colner
10/06/2021, 3:32 PMAndor Tóth
10/06/2021, 4:33 PMAndor Tóth
10/06/2021, 5:05 PMShaoyi Zhang
10/06/2021, 5:38 PMflow.run_config = ECSRun(labels=['prod'],
image='xxxxxxxx',
env={"EXTRA_PIP_PACKAGES": "retry"}
)
I was expecting that although the Kubernetes agent has the label “prod”, prefect scheduler should still only route flow runs to the ECS agent. However, looks like that’s not the case. Error:
Flow run ad4bce7b-29cb-4e42-a8c3-eeb9a75f7999 has a `run_config` of type `ECSRun`, only `KubernetesRun` is supported
Abhas P
10/06/2021, 8:38 PMMatt Alhonte
10/06/2021, 10:17 PMMarwan Sarieddine
10/07/2021, 2:14 AMRyan Sattler
10/07/2021, 3:42 AMMatthias Roels
10/07/2021, 8:51 AMSam Barger
10/07/2021, 12:10 PMZheng Xie
10/07/2021, 2:24 PMAkiva Wolberg
10/07/2021, 2:54 PMconn = psycopg2.connect()
from a task back to the flow, and then use it as an argument in another task?
• if so, should I consider separating out the SSH and database connections into separate tasks?
• using PrefectSecret appropriately --- I currently have a .env
file and a task to get the environment variables and then use the results as a parameter in my other functions, but I suspect this isn't quite right.Zheng Xie
10/07/2021, 3:53 PMAdam Brusselback
10/07/2021, 6:01 PMVamsi Reddy
10/07/2021, 6:35 PMFailed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n AttributeError("Can\'t get attribute \'RandomNum\' on <module \'prefect.__main__\'
Vamsi Reddy
10/07/2021, 6:35 PMVamsi Reddy
10/07/2021, 6:35 PMBrian Sorahan
10/07/2021, 7:49 PMLeon Kozlowski
10/07/2021, 8:00 PMstate_handler
) with prefect cloud, does the webhook URL have to be stored in Prefect Secrets, or can it just be available as an environment variable (my use case would be loading an eternal secret to an eks cluster)Constantino Schillebeeckx
10/07/2021, 8:15 PMon_failure
to it. The callback has (according to the docs) the following signature
def f(obj: Union[Task, Flow], state: State) -> None:
My quest is this: how to identify the failed task from the (Flow) obj. I see I can iterate over the flows' tasks, I'm just not sure how to identify the task's statekevin
10/08/2021, 12:52 AMKlemen Strojan
10/08/2021, 6:06 AMKeyError: 'The secret <some_secret> was not found. Please ensure that it was set correctly in your tenant: <https://docs.prefect.io/orchestration/concepts/secrets.html>'
Now this happens in a mapped task, only for one or two of those mapped tasks, others work fine. The secret exists and is clearly usable. Re-running the flow usually helps, however we are not enjoying that re-running part so much.
Any advice?Marko Herkaliuk
10/08/2021, 8:15 AMrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: <https://api.prefect.io/>
when flow try create other flow. Was it a common problem or a problem on our side?