Aiden Price
08/19/2021, 6:14 AMresource_manager
class, the flow times out and errors like this appear;
Error during execution of task: AttributeError("'NoneType' object has no attribute 'setup'")
Aiden Price
08/19/2021, 6:17 AMwith
statement, or perhaps that I'm calling a prefect.context.get("logger")
during the __init__
method.Kevin Kho
Aiden Price
08/19/2021, 10:32 PMAiden Price
08/19/2021, 10:32 PM@resource_manager
class AzureEventHubProducerClient:
"""Build an Eventhub producer client"""
def __init__(self, conn_str: str, eventhub_name: str) -> None:
self.conn_str = conn_str
self.eventhub_name = eventhub_name
def setup(self) -> EventHubProducerClient:
logger = prefect.context.get("logger")
<http://logger.info|logger.info>(f"Connecting to {self.eventhub_name}")
return EventHubProducerClient.from_connection_string(
self.conn_str, eventhub_name=self.eventhub_name, retry_total=1
)
def cleanup(self, client: EventHubProducerClient) -> None:
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Closing Azure Eventhub Producer client")
client.close()
Aiden Price
08/19/2021, 10:34 PM# Set up clients
with Oauth2_HTTPClient(
base_url=coldstream_url,
token_url=token_url,
client_id=client_id,
client_secret=client_secret,
scope=scope,
) as http_client, AzureEventHubProducerClient(
conn_str=eventhub_conn_str, eventhub_name=eventhub_name
) as eventhub_client, PostgresEngine(
timescale_conn_str
) as timescale:
all_gaps = fetch_gaps(
engine=timescale,
start_time=start,
end_time=end,
min_gap_duration=min_gap,
)
Aiden Price
08/19/2021, 10:34 PMKevin Kho
with
?Aiden Price
08/19/2021, 10:37 PMAiden Price
08/19/2021, 10:39 PMKevin Kho
resource_manager
is for hardware oriented stuff. I am not 100% sure, I’ll need to talk to someone tomorrow. Is it a specific class failing? Or any of them? Or is it the one you posted?Aiden Price
08/19/2021, 10:43 PMKevin Kho
Aiden Price
08/19/2021, 10:46 PMAiden Price
08/20/2021, 5:53 AMresource_managers
but the infinite loop is causing other tasks to fail and report their errors?Kevin Kho
Aiden Price
08/21/2021, 1:42 AMError during execution of task: AttributeError("'NoneType' object has no attribute 'setup'")
in my resource_managers
. So, is there an out of memory error or a zombie killer that might be reacting to the infinite loop by killing the resource_manager
?Kevin Kho
Aiden Price
08/22/2021, 4:07 AM