Sushma Adari
06/01/2022, 6:39 PMNo heartbeat detected from the remote task; marking the run as failed.
Chris Reuter
06/01/2022, 6:50 PMPatrick Tan
06/01/2022, 8:05 PMwith Flow("LiveLots-ETL-Parent", storage=S3(bucket=config_bucket,
client_options={
"aws_access_key_id":aws_access_key_id,
"aws_secret_access_key":aws_secret_access_key,
"aws_session_token":aws_session_token},)) as f:
Local agent is running on container as ECS task. The ECS task is attached with permission to access all S3 bucket.
{
"Statement": [
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::*"
},
{
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::wp-livelots-pt/*",
"arn:aws:s3:::wp-live-lots/*"
]
}
],
"Version": "2012-10-17"
}
Initially flow is running without issue. After one hour, I am getting
Failed to load and execute flow run: ClientError('An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.')
Looks like ECS task is using the credentials when I registered flow. Please helpFuETL
06/01/2022, 8:45 PMDylan
06/01/2022, 9:33 PMDK
06/01/2022, 9:41 PMRainer Schülke
06/02/2022, 9:59 AMEinar Ellingsen
06/02/2022, 12:23 PMmail_settings = {
"mail_title": "Title",
"mail_recipient": "<mailto:recipient.1@enova.no|recipient.1@enova.no>,recipient.2@enova.no",
"mail_sender": "<mailto:sender@enova.no|sender@enova.no>",
"mail_smtp_server": "<http://smtp-mail.outlook.com|smtp-mail.outlook.com>",
"mail_smtp_port": 587,
"mail_smtp_type": "STARTTLS",
}
with case(new_file, True):
email_sent = email(
subject=mail_settings["mail_title"],
email_to=mail_settings["mail_recipient"],
email_from=mail_settings["mail_sender"],
smtp_server=mail_settings["mail_smtp_server"],
smtp_port=mail_settings["mail_smtp_port"],
smtp_type=mail_settings["mail_smtp_type"],
msg_plain=email_body,
attachments=[most_recent_file["local_path"]],
)
Florian Guily
06/02/2022, 12:38 PMRio McMahon
06/02/2022, 3:47 PMConnor Skennerton
06/02/2022, 4:14 PMDharit Sura
06/02/2022, 4:22 PMJohn Mizerany
06/02/2022, 4:23 PMS3Upload
module in prefect is it possible to specify the content-type?Connor Skennerton
06/02/2022, 4:31 PMFailed to retrieve task state with error: ClientError([{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}])
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/engine/cloud/task_runner.py", line 154, in initialize_run
task_run_info = self.client.get_task_run_info(
File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 1479, in get_task_run_info
result = self.graphql(mutation) # type: Any
File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 473, in graphql
raise ClientError(result["errors"])
prefect.exceptions.ClientError: [{'path': ['get_or_create_task_run_info'], 'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'extensions': {'code': 'INTERNAL_SERVER_ERROR', 'exception': {'message': 'Expected type UUID!, found ""; Could not parse UUID: ', 'locations': [{'line': 2, 'column': 101}], 'path': None}}}]
Marcin Grzybowski
06/02/2022, 4:35 PMroot@3b776c4684ea:/prefect/jobs# python3 simple.py
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/asgi_lifespan/_concurrency/asyncio.py", line 17, in wait
await self._event.wait()
File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
await fut
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Jessica Smith
06/02/2022, 4:53 PMOctopus
06/02/2022, 6:22 PMJessica Smith
06/02/2022, 6:32 PM{
"errors": [
{
"path": [
"flow_run",
0,
"id"
],
"message": "Cannot return null for non-nullable field flow_run.id.",
"extensions": {
"code": "INTERNAL_SERVER_ERROR"
}
}
],
"data": null
}
I don't know how these values can be null if they...can't be null.Patrick Tan
06/02/2022, 6:33 PMMatt Alhonte
06/02/2022, 11:54 PMroot
?)wonsun
06/03/2022, 1:31 AMpath
must be provided to show where flow .py
file is stored.`
My flow code like this!
with Flow('flow_step1', storage=Local()) as flow:
engine = create_engine('<mysql+pymysql://wonsun.jeong>:~~~~~~~~~~~~~~~', echo=True)
connection = engine.connect()
before = check_db()
after = check_storage()
answer = coin_inspection(after, before)
with case(answer, False):
new_datas = action_if_false()
raw_extract = each_file_info.map(new_datas)
final = each_ecg_meta.map(raw_extract)
with case(answer, True):
action_if_true()
flow.storage = Local(stored_as_script=True)
flow.register(project_name='try')
flow.run()
# flow.visualize()
This python file name is 'flow_step1.py'. What's the problem??
Also, most weird thing is window(local) already run 'flow_step1' and create result files but prefect web UI can't detect this flow!
I'm so confused because i don't understand exactly Flow, Register, Agent, Server, Local, Path.. in Prefect. Help me please.. 🤯Tarek
06/03/2022, 11:25 AMYD
06/03/2022, 2:30 PMFrancis Ho
06/03/2022, 3:54 PMallow-errors
equivalent of jupyter-nbconvert (https://nbconvert.readthedocs.io/en/latest/config_options.html)flavienbwk
06/03/2022, 4:14 PMMax Mose
06/03/2022, 4:34 PM9a3a44b8-efeb-48bf-b07c-22a90ca9b63e
), the log messages state: No heartbeat detected from the remote task; marking the run as failed.
for some task, but the flow is still in a running state. The issue is intermittent, 1f570a06-56c8-46e4-acf5-3e0cf6d12b75
is a flow where the heartbeat marked the same task as failed and it also successfully marked the flow as failed. Any insight into what may be going on here would be much appreciated!Horatiu Bota
06/03/2022, 4:55 PMSushma Adari
06/03/2022, 6:31 PMAnat Tal Gagnon
06/03/2022, 7:36 PMAnat Tal Gagnon
06/03/2022, 7:36 PM