I m trying to get `prefect agent start` to connect to Azure Prefect Community #ask-community

I’m trying to get `prefect agent start` to connect...

Ziyao Wei

04/11/2020, 5:47 PM

I’m trying to get

prefect agent start

to connect to Azure - I am passing in a

AZURE_STORAGE_CONNECTION_STRING

but it complains:

Copy code

Unexpected error: HTTPError('400 Client Error: Bad Request for url: http://<URL>:4200/graphql/alpha')
Traceback (most recent call last):
  File "/home/<username>/.local/lib/python3.7/site-packages/prefect/client/secrets.py", line 121, in get
    value = secrets[self.name]
KeyError: 'AZ_CREDENTIALS'

Jeremiah

04/11/2020, 5:56 PM

@Ziyao Wei are there additional lines in that traceback? I think the default value for the credentials may be taking precedence over your `CONNECTION_STRING`and I’m trying to see where that would happen.

Ziyao Wei

04/11/2020, 5:58 PM

Copy code

April 11th 2020 at 1:46:15pm | prefect.CloudTaskRunner
ERROR 
Unexpected error: HTTPError('400 Client Error: Bad Request for url: http://<URL>:4200/graphql/alpha')
Traceback (most recent call last):
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/client/secrets.py", line 121, in get
    value = secrets[self.name]
KeyError: 'AZ_CREDENTIALS'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/task_<http://runner.py|runner.py>", line 925, in get_task_run_state
    state._result.store_safe_value()
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/result/base.py", line 127, in store_safe_value
    value = self.result_handler.write(self.value)
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/result_handlers/azure_result_<http://handler.py|handler.py>", line 110, in write
    self.service.create_blob_from_text(
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/result_handlers/azure_result_<http://handler.py|handler.py>", line 75, in service
    self.initialize_service()
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/engine/result_handlers/azure_result_<http://handler.py|handler.py>", line 59, in initialize_service
    azure_credentials = Secret(self.azure_credentials_secret).get()
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/client/secrets.py", line 130, in get
    variables=dict(name=self.name),
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/client/client.py", line 220, in graphql
    token=token,
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/client/client.py", line 180, in post
    token=token,
  File "/home/<USER>/.local/lib/python3.7/site-packages/prefect/client/client.py", line 317, in _request
    response.raise_for_status()
  File "/conda/envs/rapids/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://<URL>:4200/graphql/alpha

Ziyao Wei

04/11/2020, 5:58 PM

The whole shebang

Jeremiah

04/11/2020, 5:58 PM

Thanks!

Ziyao Wei

04/11/2020, 5:59 PM

The command is

AZURE_STORAGE_CONNECTION_STRING=XXXXXX prefect agent start

Ziyao Wei

04/11/2020, 5:59 PM

Thank you!

Jeremiah

04/11/2020, 6:00 PM

And for your flow, you’re passing an

Azure

storage is that correct?

Ziyao Wei

04/11/2020, 6:02 PM

I believe so -

Copy code

storage = Azure(container="prefect-flows")
with Flow("Hello World from Azure!", storage=storage) as flow:
    simple_task_test()

flow.register()

Jeremiah

04/11/2020, 6:09 PM

Ok — thanks for bringing this up! First off, I can confirm this is a bug related to how we are handling an old way of authenticating, and I think we will need to strip out in the next release, because it’s frustrating and plainly not working correctly. To work around it, could you please try adding

flow.result_handler.azure_credentials_secret=None

just prior to registering the flow?

Jeremiah

04/11/2020, 6:14 PM

I opened a bug report here and I anticipate this will be part of a larger push to streamline authentication

Jeremiah

04/11/2020, 6:19 PM

In addition, those code blocks in the docs should be updated shortly!

Ziyao Wei

04/11/2020, 6:30 PM

Thanks for looking into this! Will try the workaround now

Ziyao Wei

04/11/2020, 6:43 PM

Seems to be working, thanks! Now I need to figure out which version of the Azure Python package to use…

Ziyao Wei

04/11/2020, 7:22 PM

I do need to pass in a handler explicitly otherwise it’s not there -

Copy code

result_handler = AzureResultHandler(container=container, azure_credentials_secret=None)
with Flow("Hello World from Azure!", storage=storage, result_handler=result_handler) as flow:
    simple_task_test()

Open in Slack

Previous Next