prefect.client.get_client imports orjson and I am ...
# prefect-community
n
prefect.client.get_client imports orjson and I am having a hard time putting that in a lambda. This is the file structure I am looking at, this will eventually be all zipped up and put in an AWS lambda.
Copy code
root
├── orjson
|   ├── __pycache__
|   ├── __init__.py
|   ├── __init__.pyi
|   ├── orjson.cp39-win_amd64.pyd
|   └── py.typed
└── service.py
In service.py I am trying to import import prefect.client that then imports orjson
Copy code
import orjson
This is what init.py looks like
Copy code
from .orjson import *

__doc__ = orjson.__doc__
if hasattr(orjson, "__all__"):
    __all__ = orjson.__all__
however importing orjson throws this error:
Copy code
File "C:\work\testing_service\sand02-cumulus-pipeline\service.py", line 7, in <module>
    from prefect.client import get_client
  File "C:\work\testing_service\sand02-cumulus-pipeline\prefect\__init__.py", line 24, in <module>
    from prefect.orion.schemas.states import State
  File "C:\work\testing_service\sand02-cumulus-pipeline\prefect\orion\schemas\__init__.py", line 1, in <module>
    from . import states, schedules, data, core, sorting, filters, responses, actions
  File "C:\work\testing_service\sand02-cumulus-pipeline\prefect\orion\schemas\states.py", line 14, in <module>
    from prefect.orion.schemas.data import DataDocument
  File "C:\work\testing_service\sand02-cumulus-pipeline\prefect\orion\schemas\data.py", line 8, in <module>
    from prefect.orion.utilities.schemas import PrefectBaseModel
  File "C:\work\testing_service\sand02-cumulus-pipeline\prefect\orion\utilities\schemas.py", line 12, in <module>
    import orjson
  File "C:\work\testing_service\sand02-cumulus-pipeline\orjson\__init__.py", line 1, in <module>
    from .orjson import *
ModuleNotFoundError: No module named 'orjson.orjson'
I have no clue why this import doesn't work because if I simply have orjson in an anaconda environment and import it it works fine. Here is init.pyi if that is of any use:
Copy code
import json
from typing import Any, Callable, Optional, Union

__version__: str

def dumps(
    __obj: Any,
    default: Optional[Callable[[Any], Any]] = ...,
    option: Optional[int] = ...,
) -> bytes: ...
def loads(__obj: Union[bytes, bytearray, memoryview, str]) -> Any: ...

class JSONDecodeError(json.JSONDecodeError): ...
class JSONEncodeError(TypeError): ...

OPT_APPEND_NEWLINE: int
OPT_INDENT_2: int
OPT_NAIVE_UTC: int
OPT_NON_STR_KEYS: int
OPT_OMIT_MICROSECONDS: int
OPT_PASSTHROUGH_DATACLASS: int
OPT_PASSTHROUGH_DATETIME: int
OPT_PASSTHROUGH_SUBCLASS: int
OPT_SERIALIZE_DATACLASS: int
OPT_SERIALIZE_NUMPY: int
OPT_SERIALIZE_UUID: int
OPT_SORT_KEYS: int
OPT_STRICT_INTEGER: int
OPT_UTC_Z: int
b
In cases like this, it may be useful to print
sys.path
and see how that's different when the import succeeds ("in an anaconda environment") and when it fails. So, right before the failing
from .orjson import *
, just stick
import sys; print(sys.path)
. It may also be useful to add
import os; print(os.environ.get('PYTHONPATH'))
.
Also in this case, it may be useful to add
print(__name__)
, since you're using a relative import. You're doing
from .orjson import *
, and the error is "ModuleNotFoundError: No module named 'orjson.orjson'", so in that case it seems like
__name__
is probably
orjson
. But when you're running in an anaconda environment, perhaps
__name__
is different in that file.
Aaaaand one last thing to keep in mind is that
sys.path
is affected by how you call the
python
binary (if that's how you're running your code). Specifically, if you do
python module1/module2.py
, it will actually add "module1" to
sys.path
but not the current directory. Whereas if you do
python -m module1.module2
, it does not add "module1" to
sys.path
, and does add the current directory (which usually shows up in
sys,path
as the empty string,
''
).