Hi everyone! I hope it's a correct place to ask. So, I have the following situation. I use prefect block to store google service account key (in json). But the problem is that the downstream code that uses it requires a path to file (

langchain.document_loaders.GoogleDriveLoader

for that matters) So I cannot just load the block and pass it directly The workaround that I'm using is to 1. load the block 2. get the secret value 3. write it to a temp file 4. use it

temp_creds = tempfile.NamedTemporaryFile()
    creds_dict = GcpCredentials.load("google-drive-creds").service_account_info.get_secret_value()
    with open(temp_creds.name, 'w') as f_out:
        json.dump(creds_dict, f_out)
    loader = GoogleDriveLoader(service_account_key=Path(temp_creds.name),
                               document_ids=document_ids)
    raw_docs = loader.load()
    temp_creds.close()

To me this solution seems extremely ugly. And my question is - is there a way to do it better?

hmm this strikes me as a restriction of the

GoogleDriveLoader

class, since it requires you pass a path to the credentials file instead of also accepting the value i might use a context manager for the temp file

creds_dict = GcpCredentials.load("google-drive-creds").service_account_info.get_secret_value()

with tempfile.NamedTemporaryFile(mode='w+') as temp_creds:
    json.dump(creds_dict, temp_creds)
    loader = GoogleDriveLoader(service_account_key=Path(temp_creds.name), document_ids=document_ids)

raw_docs = loader.load()

but is there something on the prefect side you'd like to see here?

hmm this strikes me as a restriction of the

GoogleDriveLoader

class

I agree

cool! sounds good also for AI tooling, you might be in interested https://github.com/PrefectHQ/marvin its how @Marvin works 🙂 we dont have all the Loaders / vectorstore support but lots of nifty tools and abstractions for working with LLMs we're releasing a big rewrite very soon, you can try it out with

❯ pip install git+<https://github.com/PrefectHQ/marvin.git>

Wow! looks impressive! I'll certainly give it a try!