Philip MacMenamin
04/19/2022, 8:12 PMexport PREFECT__LOGGING__LEVEL=INFO
I see:
python3 tmp/shell_task.py
[2022-04-19 21:09:30+0100] INFO - prefect.FlowRunner | Beginning Flow run for 'My Flow'
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'MyShellTask': Starting task run...
[2022-04-19 21:09:30+0100] INFO - prefect.MyShellTask | lsto echo
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'MyShellTask': Finished task run for task with final state: 'Success'
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'problem': Starting task run...
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | FAIL signal raised: FAIL('Oh no!')
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | Task 'problem': Finished task run for task with final state: 'Failed'
[2022-04-19 21:09:30+0100] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
If I
export PREFECT__LOGGING__LEVEL=ERROR
I see nothing. Ideally I'd like to only see messages about broken stuff. Ideas?Nate
04/19/2022, 8:27 PMFAIL
signal in this case), then its going to be INFO
level, as you can see in your logs
[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | FAIL signal raised: FAIL('Oh no!')
any unexpected errors should show up as ERROR
level logs - is that helpful?Philip MacMenamin
04/19/2022, 8:30 PMexport PREFECT__LOGGING__LEVEL=ERROR
I see nothing. WF runs out, reports nothing at all.Anna Geller
04/19/2022, 11:07 PM[2022-04-19 21:09:30+0100] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
this is much easier to configure in Prefect 2.0 - perhaps a good reason to run your flow on 2.0 already? check out this examplePhilip MacMenamin
04/20/2022, 2:03 PM[2022-04-19 21:09:30+0100] INFO - prefect.TaskRunner | FAIL signal raised: FAIL('Oh no!')
Is is not expected behaviour that raising a FAIL would show up on an error level log?Anna Geller
04/20/2022, 2:12 PMPhilip MacMenamin
04/20/2022, 2:16 PM[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'log': Finished task run for task with final state: 'Success'
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'update_adoc': Starting task run...
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'update_adoc': Finished task run for task with final state: 'Mapped'
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'update_adoc[0]': Starting task run...
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'update_adoc[0]': Finished task run for task with final state: 'Success'
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'create_brt_command': Starting task run...
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'create_brt_command': Finished task run for task with final state: 'Mapped'
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'create_brt_command[0]': Starting task run...
[2022-04-19 12:56:09-0600] INFO - prefect.TaskRunner | Task 'create_brt_command[0]': Finished task run for task with final state: 'Success'
[2022-04-19 12:56:10-0600] INFO - prefect.TaskRunner | Task 'log': Starting task run...
which gets to be crippling if you're mapping over a lot of stuff. I don't want to be told about all of this, I just want to know if anything broke, and ideally some kind of clue about what broke.from prefect import flow, task
from logging import getLogger
logger = getLogger("my-logger")
logger.setLevel("INFO")
@task
def my_task():
<http://logger.info|logger.info>("Hello from the task")
@flow
def my_flow():
<http://logger.info|logger.info>("Hello from the flow")
my_task()
my_flow()
running with
PREFECT_LOGGING_LEVEL=WARNING PREFECT_LOGGING_EXTRA_LOGGERS=my-logger python3 flow2.py
gives me:
Traceback (most recent call last):
File "flow2.py", line 1, in <module>
from prefect import flow, task
ImportError: cannot import name 'flow' from 'prefect' (/home/macmenaminpe/.local/lib/python3.8/site-packages/prefect/__init__.py)
cat flow.py
from prefect import Flow, task
from logging import getLogger
logger = getLogger("my-logger")
logger.setLevel("INFO")
@task
def my_task():
<http://logger.info|logger.info>("Hello from the task")
with Flow("My Flow") as f:
<http://logger.info|logger.info>("Hello from the flow")
my_task()
f.run()
works, but gives me:
$ PREFECT_LOGGING_LEVEL=WARNING PREFECT_LOGGING_EXTRA_LOGGERS=my-logger python3 flow.py
[2022-04-20 15:50:10+0100] INFO - prefect.FlowRunner | Beginning Flow run for 'My Flow'
[2022-04-20 15:50:10+0100] INFO - prefect.TaskRunner | Task 'my_task': Starting task run...
[2022-04-20 15:50:10+0100] INFO - prefect.TaskRunner | Task 'my_task': Finished task run for task with final state: 'Success'
[2022-04-20 15:50:10+0100] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
Ie, same as previous, too much logging.$ prefect version
1.2.0
Nate
04/20/2022, 3:01 PMprefect>=2.0
syntax and imports, which aren't valid for prefect==1.x.x
Philip MacMenamin
04/20/2022, 3:02 PMAnna Geller
04/20/2022, 3:02 PMI just want to know if anything broke, and ideally some kind of clue about what broke.The best way of solving that problem would be through state handlers or failure notifications that you could configure through Automation. This blog post discusses both
Philip MacMenamin
04/20/2022, 3:14 PMAnna Geller
04/20/2022, 3:38 PMPhilip MacMenamin
04/20/2022, 3:44 PMAnna Geller
04/20/2022, 3:52 PMPhilip MacMenamin
04/20/2022, 3:57 PMAnna Geller
04/20/2022, 4:51 PM