Vitaly Shulgin
12/18/2020, 9:11 AMnicholas
Vitaly Shulgin
12/18/2020, 1:57 PM[2020-12-18 12:07:48+0300] INFO - prefect.FlowRunner | Beginning Flow run for 'testing-etl-visitors'
[2020-12-18 12:07:48+0300] DEBUG - prefect.FlowRunner | Using executor type LocalExecutor
[2020-12-18 12:07:48+0300] DEBUG - prefect.FlowRunner | Flow 'testing-etl-visitors': Handling state change from Scheduled to Running
[2020-12-18 12:07:48+0300] INFO - prefect.TaskRunner | Task 'context_init': Starting task run...
[2020-12-18 12:07:48+0300] DEBUG - prefect.TaskRunner | Task 'context_init': Handling state change from Pending to Running
[2020-12-18 12:07:48+0300] DEBUG - prefect.TaskRunner | Task 'context_init': Calling task.run() method...
[2020-12-18 12:07:50+0300] DEBUG - prefect.context_init | Initializing context ...
[2020-12-18 12:07:50+0300] DEBUG - prefect.context_init | Initializing injector ...
[2020-12-18 12:07:50+0300] DEBUG - prefect.context_init | Initialized injector successfully
[2020-12-18 12:07:50+0300] DEBUG - prefect.context_init | Succeeded with auth for firebase <<https://onedebot.firebaseio.com/visitors.json>>
[2020-12-18 12:07:50+0300] DEBUG - prefect.context_init | Initialized context successfully
[2020-12-18 12:07:50+0300] DEBUG - prefect.TaskRunner | Task 'context_init': Handling state change from Running to Success
[2020-12-18 12:07:50+0300] INFO - prefect.TaskRunner | Task 'context_init': Finished task run for task with final state: 'Success'
[2020-12-18 12:07:50+0300] INFO - prefect.TaskRunner | Task 'visitors_extract': Starting task run...
[2020-12-18 12:07:50+0300] DEBUG - prefect.TaskRunner | Task 'visitors_extract': Handling state change from Pending to Running
[2020-12-18 12:07:50+0300] DEBUG - prefect.TaskRunner | Task 'visitors_extract': Calling task.run() method...
[2020-12-18 12:07:50+0300] DEBUG - prefect.TaskRunner | Task 'visitors_extract': Handling state change from Running to Success
[2020-12-18 12:07:50+0300] INFO - prefect.TaskRunner | Task 'visitors_extract': Finished task run for task with final state: 'Success'
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'visitors_transform': Starting task run...
[2020-12-18 12:07:51+0300] DEBUG - prefect.TaskRunner | Task 'visitors_transform': Handling state change from Pending to Running
[2020-12-18 12:07:51+0300] DEBUG - prefect.TaskRunner | Task 'visitors_transform': Calling task.run() method...
[2020-12-18 12:07:51+0300] DEBUG - prefect.TaskRunner | Task 'visitors_transform': Handling state change from Running to Success
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'visitors_transform': Finished task run for task with final state: 'Success'
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'filter_visitor_by_user_agent': Starting task run...
[2020-12-18 12:07:51+0300] DEBUG - prefect.TaskRunner | Task 'filter_visitor_by_user_agent': Handling state change from Pending to Failed
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'filter_visitor_by_user_agent': Finished task run for task with final state: 'Failed'
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'filter_final': Starting task run...
[2020-12-18 12:07:51+0300] DEBUG - prefect.TaskRunner | Task 'filter_final': Handling state change from Pending to Failed
[2020-12-18 12:07:51+0300] INFO - prefect.TaskRunner | Task 'filter_final': Finished task run for task with final state: 'Failed'
[2020-12-18 12:07:51+0300] INFO - prefect.FlowRunner | Flow run FAILED: some reference tasks failed.
[2020-12-18 12:07:51+0300] DEBUG - prefect.FlowRunner | Flow 'testing-etl-visitors': Handling state change from Running to Failed
with Flow("testing-etl-visitors") as flow:
logger = prefect.context.get("logger")
logger.setLevel(logging.DEBUG)
init_succeeded = context_init()
json_data = visitors_extract(init_succeeded)
all_visitors = visitors_transform(json_data)
filtered_visitors = filter_visitor_by_user_agent.map(all_visitors)
final_visitors = filter_final.map(filtered_visitors)
result = flow.run()
nicholas
Vitaly Shulgin
12/18/2020, 2:26 PMyiedl
, which prevents task to be executed as expectednicholas