jack
12/15/2020, 2:32 AMprint()
statements to show up in the web console at localhost:8080?
Here's my example code that prints to the terminal, but does not now up in the web console:
from prefect import task, Flow
@task
def simple_func(log_stdout=True):
print('simple print statement')
if __name__ == '__main__':
with Flow('test-logging') as flow:
simple_func()
flow.register('test-log')
flow.run()
nicholas
12/15/2020, 2:34 AMjack
12/15/2020, 2:35 AMlog_to_cloud = true
in config.toml or not.log_to_server
option? I'm using the server backend[2020-12-15 02:37:16+0000] CRITICAL - CloudHandler | Failed to write log with error: 400 Client Error: Bad Request for url: <http://localhost:4200/graphql>
The following error messages were provided by the GraphQL server:
INTERNAL_SERVER_ERROR: Variable "$input" got invalid value null at
"input.logs[0].flow_run_id"; Expected non-nullable type UUID! not to be null.
The GraphQL query was:
mutation($input: write_run_logs_input!) {
write_run_logs(input: $input) {
success
}
}
The passed variables were:
{"input": {"logs": [{"flow_run_id": null, "task_run_id": null, "timestamp": "2020-12-15T02:37:15.644721+00:00", "name": "prefect.FlowRunner", "message": "Beginning Flow run for 'test-alteryx-logging'", "level": "INFO", "info": {"msg": "Beginning Flow run for 'test-alteryx-logging'", "levelno": 20, "pathname": "/home/vagrant/p/ts-prefect-1.0/env/lib64/python3.7/site-packages/prefect/engine/flow_runner.py", "filename": "flow_runner.py", "module": "flow_runner", "exc_info": null, "exc_text": null, "stack_info": null, "lineno": 224, "funcName": "run", "msecs": 644.721269607544, "relativeCreated": 2320.8816051483154, "thread": 140260445312832, "threadName": "MainThread", "processName": "MainProcess", "process": 14188, "asctime": "2020-12-15 02:37:15+0000"}}, {"flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "task_run_id": "b2b390d0-bf27-42dc-9c50-2b076563fae6", "timestamp": "2020-12-15T02:37:15.929799+00:00", "name": "prefect.TaskRunner", "message": "Task 'simple_func': Starting task run...", "level": "INFO", "info": {"msg": "Task 'simple_func': Starting task run...", "levelno": 20, "pathname": "/home/vagrant/p/ts-prefect-1.0/env/lib64/python3.7/site-packages/prefect/engine/task_runner.py", "filename": "task_runner.py", "module": "task_runner", "exc_info": null, "exc_text": null, "stack_info": null, "lineno": 237, "funcName": "run", "msecs": 929.7986030578613, "relativeCreated": 2605.958938598633, "thread": 140260445312832, "threadName": "MainThread", "processName": "MainProcess", "process": 14188, "flow_name": "test-alteryx-logging", "flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "task_name": "simple_func", "task_slug": "simple_func-1", "task_run_id": "b2b390d0-bf27-42dc-9c50-2b076563fae6", "asctime": "2020-12-15 02:37:15+0000"}}, {"flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "task_run_id": "b2b390d0-bf27-42dc-9c50-2b076563fae6", "timestamp": "2020-12-15T02:37:16.045748+00:00", "name": "prefect.TaskRunner", "message": "Task 'simple_func': Finished task run for task with final state: 'Success'", "level": "INFO", "info": {"msg": "Task 'simple_func': Finished task run for task with final state: 'Success'", "levelno": 20, "pathname": "/home/vagrant/p/ts-prefect-1.0/env/lib64/python3.7/site-packages/prefect/engine/task_runner.py", "filename": "task_runner.py", "module": "task_runner", "exc_info": null, "exc_text": null, "stack_info": null, "lineno": 332, "funcName": "run", "msecs": 45.74751853942871, "relativeCreated": 2721.9078540802, "thread": 140260445312832, "threadName": "MainThread", "processName": "MainProcess", "process": 14188, "flow_name": "test-alteryx-logging", "flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "task_name": "simple_func", "task_slug": "simple_func-1", "task_run_id": "b2b390d0-bf27-42dc-9c50-2b076563fae6", "asctime": "2020-12-15 02:37:16+0000"}}, {"flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "task_run_id": null, "timestamp": "2020-12-15T02:37:16.048198+00:00", "name": "prefect.FlowRunner", "message": "Flow run SUCCESS: all reference tasks succeeded", "level": "INFO", "info": {"msg": "Flow run SUCCESS: all reference tasks succeeded", "levelno": 20, "pathname": "/home/vagrant/p/ts-prefect-1.0/env/lib64/python3.7/site-packages/prefect/engine/flow_runner.py", "filename": "flow_runner.py", "module": "flow_runner", "exc_info": null, "exc_text": null, "stack_info": null, "lineno": 685, "funcName": "determine_final_state", "msecs": 48.19774627685547, "relativeCreated": 2724.358081817627, "thread": 140260445312832, "threadName": "MainThread", "processName": "MainProcess", "process": 14188, "flow_name": "test-alteryx-logging", "flow_run_id": "9e720791-b7c1-406e-a773-f3ff23e4cb01", "asctime": "2020-12-15 02:37:16+0000"}}]}}
[2020-12-15 02:37:16+0000] CRITICAL - CloudHandler | Unable to write logs to Prefect Cloud
[2020-12-15 02:37:16+0000] CRITICAL - CloudHandler | Unable to write logs to Prefect Cloud
[2020-12-15 02:37:16+0000] CRITICAL - CloudHandler | Unable to write logs to Prefect Cloud
log_to_cloud
to true, running the task throws an error:nicholas
12/15/2020, 2:38 AMlog_to_cloud
will send to either Server or Cloud (whichever it's running against)flow.run()
)jack
12/15/2020, 2:48 AMflow.run()
I don't expect the log to show up in the browser.
But after registering the flow and then running it from the browser, that's when I expect the log to show up in the browser. (But it doesn't)nicholas
12/15/2020, 2:50 AMjack
12/15/2020, 2:51 AMnicholas
12/15/2020, 2:57 AMlog_stdout
as a keyword to your task, instead of to the task decorator... your above code should be:
@task(log_stdout=True)
def simple_func():
print('simple print statement')
jack
12/15/2020, 3:08 AMnicholas
12/15/2020, 3:10 AMlog_to_cloud
still set to true in your config.toml?jack
12/15/2020, 3:13 AM[logging]
log_to_cloud = true
and with
[logging]
log_to_cloud = false
and with
[logging]
#log_to_cloud = true
They are give the same result: the log goes to the web browser.nicholas
12/15/2020, 3:26 AMlog_stdout
kwarg you'll need to reregister - did you already do that?jack
12/15/2020, 1:18 PMlog_stdout
kwarg in place, but just changed the config.toml.log_to_cloud
does so I can add it to the documentation. And for me the logs are showing up regardless of the log_to_cloud
setting.nicholas
12/15/2020, 5:13 PMlog_to_cloud
will determine if any logs are sent to your Server... however changes to that will require that you restart your agent.log_stdout
insofar as that uses the Prefect logger in generaljack
12/15/2020, 5:29 PMfrom prefect import task, Flow
@task(log_stdout=True)
def simple_func():
print('simple print statement')
if __name__ == '__main__':
with Flow('simple-print-statement') as flow:
simple_func()
flow.register('test-log')
flow.run()
and here is my ~/.prefect/config.toml
[logging]
log_to_cloud = false
extra_loggers = "['clone', 'special']"
and here is the log in the browser (which probably isn't expected to be there)nicholas
12/15/2020, 5:39 PMjack
12/15/2020, 5:46 PM[logging]
in config.toml. Upon restarting the agent, it raises an exception inside /env/lib64/python3.7/site-packages/toml/decoder.py
So it appears to be loading the config file.nicholas
12/15/2020, 5:52 PMjack
12/15/2020, 5:55 PMPREFECT__LOGGING__LOG_TO_CLOUD=false prefect agent local start
The flow ran about ten seconds after the agent was started, and it still has the simple print statement
in the logs in the browsernicholas
12/15/2020, 5:58 PMjack
12/15/2020, 6:15 PMnicholas
12/15/2020, 6:15 PMjack
12/15/2020, 6:25 PMnicholas
12/15/2020, 6:26 PM