https://prefect.io logo
p

Pasha Kravtsov

05/18/2022, 10:43 PM
I have a Prefect flow that fails when it is unpickled, the error message is related to me decrypting variables in pydantic's Settings().. is there any good way to move forward and debug pickling issues like this?
k

Kevin Kho

05/18/2022, 10:45 PM
Is that Prefect 1 or Prefect 2?
p

Pasha Kravtsov

05/18/2022, 10:47 PM
1.1.0
k

Kevin Kho

05/18/2022, 10:48 PM
What is the exact error?
p

Pasha Kravtsov

05/18/2022, 10:48 PM
I'm using something similar to this:
Copy code
from pydantic import BaseSettings
from pydantic_kms_secrets import KMSSecretStr, decrypt_kms_secrets


class Settings(BaseSettings):
    env: str
    secrets_kms_key_id: str  # This model attribute must exist to decrypt secrets
    kms_secret_1: KMSSecretStr
    kms_secret_2: KMSSecretStr

    class Config:
        env_file = ".env"

settings = decrypt_kms_secrets(Settings())
which works without issue outside of Prefect in my main application
Copy code
Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n  ClientError("An error occurred (ValidationException) when calling the Decrypt operation: 1 validation error detected: Value at \'ciphertextBlob\' failed to satisfy constraint: Member must have length greater than or equal to 1")')
k

Kevin Kho

05/18/2022, 11:01 PM
Maybe you can store as a script to bypass the serialization?
p

Pasha Kravtsov

05/18/2022, 11:09 PM
is there some documentation that I could take a look at please for that?
k

Kevin Kho

05/18/2022, 11:15 PM
Yep! see this
🙏 1
2 Views