davzucky
10/29/2021, 5:12 AMdata_loader-orion_agent-1 | Starting agent...
data_loader-orion_agent-1 | Agent started! Checking for flow runs...
data_loader-orion_agent-1 | 05:02:24.615 | Submitting flow run 'ddf9cb6b-231b-48e7-983a-fcaf1df850f7'
data_loader-orion_agent-1 | 05:02:24.616 | Completed submission of flow run 'ddf9cb6b-231b-48e7-983a-fcaf1df850f7'
data_loader-orion_agent-1 | 05:02:28.397 | Flow run 'ddf9cb6b-231b-48e7-983a-fcaf1df850f7' exited with exception: Abort('This run has already terminated.')
and the job doesn't do anything.
Connecting to the container and runner the command
prefect deployment execute load-data/schedulesLoadData
execute the workflow which finish with success
05:04:56.465 | Creating run for flow 'load-data'...
05:04:56.570 | Beginning flow run 'illustrious-nuthatch' for flow 'load-data'...
05:04:56.571 | Starting executor SequentialExecutor...
05:04:56.571 | Flow run 'illustrious-nuthatch' received parameters {'source_path': '/source', 'destination_path': '<s3://credit-risk/raw_data>', 'fs_type_source': 'file', 'fs_type_destination': 'file', 'fs_kwarg_source': {}, 'fs_kwarg_destination': {'client_kwargs': {'endpoint_url': '<http://minio:9000>', 'aws_access_key_id': 'admin', 'aws_secret_access_key': 'welcome123'}}}
05:04:56.628 | Executing flow 'load-data' for flow run 'illustrious-nuthatch'...
05:04:56.629 | Calling loadData('/source', '<s3://credit-risk/raw_data>', 'file', 'file', {}, {'client_kwargs': {'endpoint_url': 'http...)
05:04:56.768 | Submitting task run 'ls_files-c965bedb-0' to executor...
05:04:56.814 | Task run 'ls_files-c965bedb-0' received parameters {'fs_type_source': 'file', 'fs_kwarg_source': {}, 'base_path_source': '/source'}
05:04:56.900 | Executing task 'ls_files' for task run 'ls_files-c965bedb-0'
05:04:56.901 | Calling ls_files('file', {}, '/source')
05:04:57.041 | Task run 'ls_files-c965bedb-0' finished in state Completed(message=None, type=COMPLETED, result=[{'name': '/source/sample_data.csv', 'size': 7564965, 'type': 'file', 'created': 1635483409.8989005, 'islink': False, 'mode': 33188, 'uid': 1000, 'gid': 1000, 'mtime': 1635483409.8989005}], task_run_id=c2d8299a-1586-4310-ae09-2a3aace3fb47)
05:05:00.005 | Shutting down executor SequentialExecutor...
05:05:00.089 | Flow run 'illustrious-nuthatch' finished in state Completed(message='All states completed.', type=COMPLETED, result=[Completed(message=None, type=COMPLETED, result=[{'name': '/source/sample_data.csv', 'size': 7564965, 'type': 'file', 'created': 1635483409.8989005, 'islink': False, 'mode': 33188, 'uid': 1000, 'gid': 1000, 'mtime': 1635483409.8989005}], task_run_id=c2d8299a-1586-4310-ae09-2a3aace3fb47)], flow_run_id=a37ea827-4854-49fd-a4c7-e2e6913514b1)
Trying to get more information by setting the env variable PREFECT_DEBUG_MODE to true doesn't change anything. Do you have any input to try to get more information about what is happening on the agent ?davzucky
10/29/2021, 5:18 AMZanie
Abort('This run has already terminated.')
is raised when the flow run has already been executed by another agentdavzucky
10/29/2021, 10:55 PMZanie
davzucky
10/30/2021, 3:34 AM