Tushar Gupta
11/18/2022, 5:06 PMMiremad Aghili
11/18/2022, 5:27 PMJoshua Grant
11/18/2022, 5:30 PMdataflow-ops
repo and have tried to use the ECSTask
from prefect-aws
to circumvent the issue surrounding DockerRegistry
for ECR, but I keep getting the error KeyError: "No class found for dispatch key 'ecs-task' in registry for type 'Block'"
. Prefect==2.6.8 & prefect-aws==0.1.8Tibs
11/18/2022, 6:05 PMVipul
11/18/2022, 8:23 PMScott Walsh
11/18/2022, 9:50 PMprocess
infrastructure setting, but then the sub flow is run by an agent, not the parent flows infrastructure.Michael Z
11/18/2022, 10:30 PM*prefect.triggers.all_finished*(_upstream_states_)
. This is for prefect 1 btw.Kimera Joseph
11/19/2022, 5:58 AMKimera Joseph
11/19/2022, 6:27 AMkomal azram
11/19/2022, 6:57 AMRob Fowler
11/19/2022, 10:04 AMJimmy Le
11/19/2022, 5:47 PMsuresh
11/20/2022, 7:31 AMkomal azram
11/20/2022, 9:44 AMkomal azram
11/20/2022, 10:51 AMDekel R
11/20/2022, 12:38 PMDekel R
11/20/2022, 1:54 PMHoward
11/21/2022, 2:49 AMMohit Singhal
11/21/2022, 6:27 AMMichael Holzapfel
11/21/2022, 7:46 AMSachin Ranveer
11/21/2022, 9:13 AMAn Ninh Vũ
11/21/2022, 10:16 AMFuETL
11/21/2022, 11:05 AMFAIL
, PrefectStateSignal
, PrefectSignal
) inherit from BaseException
instead of Exception
?Patrick Alves
11/21/2022, 11:56 AMprefect config set PREFECT_LOGGING_EXTRA_LOGGERS="rootlogger,loguru,root,root logger"
But it did not worked.
Any tips?Matthew Seligson
11/21/2022, 2:53 PMLennert Van de Velde
11/21/2022, 3:05 PMJames Zhang
11/21/2022, 3:24 PMERROR | prefect.agent - Failed to submit flow run 'xxxx' to infrastructure.
how can I investigate the details?Guillermo Galan
11/21/2022, 4:02 PM16:44:27.874 | INFO | prefect.infrastructure.container-instance-job - AzureContainerInstanceJob 'sftp-to-blob-ingestion': Preparing to run command '/opt/prefect/entrypoint.sh python -m prefect.engine' in container 'a53fd966-c635-4605-a8ab-4a3f487a7463' (prefecthq/prefect:2.6.7-python3.9)...
16:44:27.875 | INFO | prefect.infrastructure.container-instance-job - AzureContainerInstanceJob 'sftp-to-blob-ingestion': Waiting for container creation...
16:45:37.024 | INFO | prefect.infrastructure.container-instance-job - AzureContainerInstanceJob 'sftp-to-blob-ingestion': Running command...
16:45:37.025 | INFO | prefect.agent - Completed submission of flow run '6bb0fc44-197e-4bfd-846e-877f5dbbf0b8'
16:48:43.454 | INFO | prefect.infrastructure.container-instance-job - AzureContainerInstanceJob 'sftp-to-blob-ingestion': Completed command run.
16:48:43.455 | INFO | prefect.infrastructure.container-instance-job - AzureContainerInstanceJob 'sftp-to-blob-ingestion': Deleting container...
In the meantime the Orion UI gets frozen in 'running' state and there are no log lines indicating any error or crash.Tushar Gupta
11/21/2022, 6:21 PMMadison Schott
11/21/2022, 7:17 PMMadison Schott
11/21/2022, 7:17 PMMason Menges
11/21/2022, 7:23 PMMadison Schott
11/21/2022, 7:23 PMMason Menges
11/21/2022, 7:26 PMMadison Schott
11/21/2022, 7:33 PMMason Menges
11/21/2022, 7:34 PMMadison Schott
11/21/2022, 7:39 PMTraceback (most recent call last):
File "/Users/madisonschott/dbt_snowflake/data_pipeline_test_new_account.py", line 387, in <module>
data_pipeline_test_new_account.register(project_name="Winc_Prod")
File "/usr/local/lib/python3.9/site-packages/prefect/core/flow.py", line 1726, in register
registered_flow = client.register(
File "/usr/local/lib/python3.9/site-packages/prefect/client/client.py", line 855, in register
serialized_flow = flow.serialize(build=build) # type: Any
File "/usr/local/lib/python3.9/site-packages/prefect/core/flow.py", line 1515, in serialize
storage = self.storage.build() # type: Optional[Storage]
File "/usr/local/lib/python3.9/site-packages/prefect/storage/docker.py", line 325, in build
self._build_image(push=push)
File "/usr/local/lib/python3.9/site-packages/prefect/storage/docker.py", line 357, in _build_image
client = self._get_client()
File "/usr/local/lib/python3.9/site-packages/prefect/storage/docker.py", line 572, in _get_client
return docker.APIClient(
File "/usr/local/lib/python3.9/site-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
File "/usr/local/lib/python3.9/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
Mason Menges
11/21/2022, 7:42 PMprefect auth list-tenants
Madison Schott
11/21/2022, 7:43 PMMason Menges
11/21/2022, 7:48 PMMadison Schott
11/21/2022, 7:55 PMStep 13/13 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/data-pipeline-test-new-account.prefect"]' '(3, 9)'
---> Running in 1fe1ff1c4327
/opt/prefect/healthcheck.py:125: UserWarning: Your Docker container is using python version (3, 7), but your Flow was serialized using (3, 9); this could lead to unexpected errors in deployment.
system_check(python_version)
Beginning health checks...
Traceback (most recent call last):
File "/opt/prefect/healthcheck.py", line 130, in <module>
flows = cloudpickle_deserialization_check(flow_file_paths)
File "/opt/prefect/healthcheck.py", line 43, in cloudpickle_deserialization_check
flows.append(cloudpickle.loads(flow_bytes))
AttributeError: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from '/usr/local/lib/python3.7/site-packages/cloudpickle/cloudpickle.py'>
Mason Menges
11/21/2022, 7:57 PMMadison Schott
11/21/2022, 8:01 PMFROM prefecthq/prefect:latest
in my base image?Mason Menges
11/21/2022, 8:03 PMMadison Schott
11/21/2022, 8:03 PMMason Menges
11/21/2022, 8:05 PMMadison Schott
11/21/2022, 8:08 PMMason Menges
11/21/2022, 8:08 PMMadison Schott
11/21/2022, 10:21 PMdbt deps