I want to add azure storage for my flows. but i got an error - No module named azure. So, I am trying to run - pip install prefect[azure] -> but this is taking too long. it is running from about an hour. Is there anything I can do to get this resolved?

Oh man. I believe this has to more with the fact that pip 0.24 has a dependency resolver and the Azure packages within themselves have dependency conflicts so you get this loop of not being able to resolve packages. You can try downgrading to pip 0.23

ok. with poetry, I am getting a different error

Doesn’t poetry just use pip though? Not sure.

Try installing just

pip install azure-storage-blob

and try rerunning your flow

ok

did that work for you? I created a PR to separate the ML requirements https://github.com/PrefectHQ/prefect/pull/5632

no, it didn't. i tried adding other libraries as well. including azure-core. but i am still getting No module named azure error

what is your python version? the azure library might have minimum

python 3.9.9

I can’t find it in the

setup.py

of their repo but I suspect your Python might be too high. Could you try 3.8?

Can you double check whether your python path is correct e.g.

which python

and azure was installed into the proper environment

which pip

pip list

(azure-test) [Fri 1 11:46] prefect: pip install azure-core azure-storage-blob
Collecting azure-core
  Using cached azure_core-1.23.1-py3-none-any.whl (178 kB)
Collecting azure-storage-blob
  Using cached azure_storage_blob-12.11.0-py3-none-any.whl (346 kB)
Collecting requests>=2.18.4
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting six>=1.11.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting typing-extensions>=4.0.1
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Collecting msrest>=0.6.21
  Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB)
Collecting cryptography>=2.1.4
  Using cached cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl (4.7 MB)
Collecting cffi>=1.12
  Using cached cffi-1.15.0-cp39-cp39-macosx_11_0_arm64.whl (173 kB)
Collecting requests-oauthlib>=0.5.0
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting isodate>=0.6.0
  Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting oauthlib>=3.0.0
  Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Installing collected packages: certifi, urllib3, typing-extensions, six, pycparser, oauthlib, idna, charset-normalizer, requests, isodate, cffi, requests-oauthlib, cryptography, azure-core, msrest, azure-storage-blob
Successfully installed azure-core-1.23.1 azure-storage-blob-12.11.0 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12 cryptography-36.0.2 idna-3.3 isodate-0.6.1 msrest-0.6.21 oauthlib-3.2.0 pycparser-2.21 requests-2.27.1 requests-oauthlib-1.3.1 six-1.16.0 typing-extensions-4.1.1 urllib3-1.26.9
(azure-test) [Fri 1 11:46] prefect: python3
Python 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:06)
[Clang 11.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import azure
>>> from azure.storage import blob
>>>

this is the toml file. we add packages using poetry add <package-name>

Can you post the traceback of the error you’re receiving?

This is the actual error.

Not really. For Azure Storage as long as

azure-storage-blob

installed it should work

ok. what else I can look for to troubleshoot this error

I am a bit confused. What is your RunConfiguration? Is this LocalRun or something like DockerRun?

So you were able to install Azure already?

no. from the time I have given storage=Azure, things are not working

Ah then you really need to get the Azure module installed locally. You can install

azure-storage-blob

separately by pip installing that

it is installed.

And you try starting an interactive terminal and doing

>>> import azure
>>> from azure.storage import blob

maybe you need to restart the local agent to reflect the new packages installed?

i have restarted everything. restarted my laptop a few minutes back

are you using virtual environments by chance? is the agent using the same virtual environment as the one with azure installed?

yes i am using virtual env. i never tried running the agent inside the virtual env. i just did and the agent is not starting because it is trying to go to localhost:4200 instead of IP:4200 my config.toml has that IP. is there some other config.toml required for virtual env?

what is your virtual env manager?

poetry

i think that’s kinda weird. the config.toml should be lived and respected because virtual environments treat it the same

trying some different ways now. passing container name in code and connection string (PREFECT__CONTEXT__SECRETS__AZURE_STORAGE_CONNECTION_STRING="<value>") as an env variable while agent setup. Now getting this error.

raise error
azure.core.exceptions.ServiceRequestError: <urllib3.connection.HTTPSConnection object at 0x000001E292AF2D90>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

I think this is specifically an Azure issue. Is this on registration or run time?

this is run time issue

When you register, it was able to upload the Flow to the blob storage?

no

Ah ok and then does that upload?

no. I am getting the above issue. getaddrinfo failed

It seems like you are unable to connect to Azure in general. Is your local machine authenticated?

I am retrying. This is the msg I get when i make the api call.

Can you try listing the blobs maybe with the Azure client to confirm you can do that? Seems like you can’t connect to Azure