https://prefect.io logo
Powered by
Title
a

Andreas Nord

11/28/2022, 11:09 AM
Hi! I want to access my Prefect Cloud secrets in a jupyter notebook: 
Secret.load("secret")
this was working on Prefect 1 but on 2 I get this error: 
RuntimeError: A 'sync_compatible' method was called from a context that was previously async but is now sync. The sync call must be changed to run in a worker thread to support sending the coroutine for 'load' to the main thread.
Any ideas? Thanks
✅ 1
j

James Sopkin

11/28/2022, 3:37 PM
Hi @Andreas Nord try loading your secrets, or any other blocks, from within a flow or task
a

Andreas Nord

11/28/2022, 3:39 PM
I get the same error if I import the task and do imported_task.fn(...)
j

James Sopkin

11/28/2022, 4:00 PM
Not sure exactly how you are using secrets in your code, but I tried running code where I loaded the secret from within a task(task_a) vs where I just imported the secret outside of a task and I got a similar error
for reference this is the 
task_a
I imported in my example 
@task
def taska(a: int):
    secret_block = Secret.load("my-secret")
    print(a + 1)
    print(secret_block.get())
a

Andreas Nord

11/28/2022, 4:06 PM
so it works to run a flow, but not a task. The use case is that I want to play around with the output of some tasks that authenticates to different data sources
So I can run prefect tasks in a python console but not in a notebook. I guess the notebooks handles async stuff differently
j

James Sopkin

11/28/2022, 4:08 PM
I've had similar issues where flows would run locally but run into async issues when deployed on cloud or when attempting to deploy flows with CI/CD
Are you using blocks when you authenticate to these data sources? For example- a snowflake creds block?
a

Andreas Nord

11/28/2022, 4:37 PM
No just plain secrets
j

James Sopkin

11/28/2022, 4:56 PM
For your use case, I would recommend loading the secret from within a flow, and retrieving the value from the task when you need to use it. 
from prefect import task


@task
def taska(a: int, secret: str):
    print(a + 1)
    print(secret.get())


@task
def taskb(secret: str):
    print(secret.get()[::-1])
from prefect import flow, task
from flow_file import taska, taskb
from prefect.blocks.system import Secret


@flow()
def myflow():
    secret = Secret.load("my-secret")
    taska(5, secret) # prints x + 1 and prints secret block
    taskb(secret) # prints secret in reverse

myflow()
Here is an example I used, but this is typically the method I would use if I was passing in, say, an api key as a secret and authenticating in a task. The reason why I would pass in the secret and not the string value of the secret into the tasks(what you get when you run the get() method), is because the string value would appear in the logs
#prefect-communityJoin Slack
Powered by