How can i use prefect server to access google clou...
# prefect-community
p
How can i use prefect server to access google cloud?
n
Hi @philip - can you expand on what you mean? Within a flow or deploying Prefect server or something else?
p
task = ShellTask()
with Flow("to_bigquery") as flow:
command = task(command='gsutil cp .... .....)
when I use
flow.run()
it can success. But when i run on Prefect Server it have an error
401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.
n
Ah I see @philip, thanks for clarifying 🙂 What seems to be happening here is that your local machine is authorized with
GCP
, which is why
flow.run
would succeed - it's running that in an authorized context. Where is your agent running? Your flow's runtime context doesn't seem to be authorized.
p
I find the problem. My agent start in the python venv, i will have the error. When I run in the original python env it can success. Thanks for your help!!
n
That makes sense! Glad I could help 😄
👍 1
p
But when i use `nohup`to run agent, i will get the same error😅
n
But only if nohup is installed in an environment, right?
Otherwise nohup should take on the context correctly, i believe
p
Excuse me, can you expand more detail?
n
Ah ok @philip you're correct that starting your agent in the python environment would cause it to lose some context including it seems the gcloud auth context.
In which case I'd recommend you explicitly authorize the context of the task you're running, by including authorization in the shell task maybe (or better by getting credentials upstream and passing them to the tasks that'll need them)
p
My agent is start at the shell, and i already authorization in the shell. But when i use nohup start the agent in the shell it still have the error. Only use prefect agent start in shell it can run success.
n
You should authorize the
ShellTask
or use a
GCP
service account credentials instead
p
Got it. Thank you!
😄 1