Salim Doost
04/14/2022, 2:53 AM404 Client Error for <http+docker://localhost/v1.41/containers/create?name=quantum-squid>: Not Found ("No such image: <account-id>.<http://dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>%22|dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>">)
However, we’re able to confirm that the image with this tag exists on EMR.
Updating an existing flow by overriding an existing image-tag leads to the following error:
KeyError: 'Task slug <task-name> is not found in the current Flow. This is usually caused by a mismatch between the flow version stored in the Prefect backend and the flow that was loaded from storage.
- Did you change the flow without re-registering it?
- Did you register the flow without updating it in your storage location (if applicable)?'
Again, we’re able to confirm in AWS ECR that the image got pushed and updated successfully. Our deployment job didn’t throw any error messages as well.
Any idea what we can do to resolve this issue?Kevin Kho
docker pull
work for the image?
2. How if the agent authenticated to pull that?
3. Is the key error during flow execution and it ends the flow?Salim Doost
04/14/2022, 3:05 AMPulling image <account-id>.<http://dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name|dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>>...
Successfully pulled image <account-id>.<http://dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name|dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>>
docker.errors.ImageNotFound: 404 Client Error for <http+docker://localhost/v1.41/containers/create?name=quantum-squid>: Not Found ("No such image: <account-id>.<http://dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>%22|dkr.ecr.ap-northeast-1.amazonaws.com/datascience-prefect:<image-tag-name>">)
3. yes, that’s the caseKevin Kho
Salim Doost
04/14/2022, 3:47 AMKevin Kho
Salim Doost
04/14/2022, 4:04 AM{
"env": null,
"type": "DockerRun",
"image": null,
"labels": [
"ec2-dockeragent"
],
"__version__": "0.15.4",
"host_config": null
}
(same for all of our flows, even those that are still running and haven’t been updated since this error occurs)
The KeyError
occurs AFAIK because it tries to access a task by name that doesn’t exist in the image. Is prefect doing any caching here maybe?Kevin Kho
Salim Doost
04/14/2022, 5:46 AMKevin Kho